LogStash

> (Data|State) Management and Processing > (Data Type|Data Structure) > Log - Logging

1 - About

LogStash is a log collector.

3 - Pipeline

A Logstash pipeline is composed of the following elements;

  • input (produce the data)
  • filter (optional, process the data)
  • output (write the data)
Advertising

4 - Getting Started

4.1 - Docker

Doc

  • Download
docker pull docker.elastic.co/logstash/logstash:7.5.1
  • Run
docker run ^
   --rm ^
   -it ^
   -v ^
   %CD%:/usr/share/logstash/pipeline/ ^
   docker.elastic.co/logstash/logstash:7.5.1 ^
   bash

where:

  • the %CD% is the current directory and will be mounted into /usr/share/logstash/pipeline/ which is the location of the pipeline configuration file logstash.conf. If there is no configuration, the (BeatInput) is used.

the log4 configuration file is located at /usr/share/logstash/config/log4j2.properties

4.2 - First pipeline

cd bin/logstash-7.5.1
# in docker
# cd /usr/share/logstash/bin/logstash
logstash -e 'input { stdin { } } output { stdout {} }'

where:

Typing hello word at the console (stdin) will produce the below message at the console (stdout)

{
       "message" => "hello world",
    "@timestamp" => 2020-01-13T14:02:43.376Z,
      "@version" => "1",
          "host" => "32621775747d"
}

Logstash adds timestamp and IP address information to the message.

Advertising

5 - Grok

6 - Documentation / Reference