A, install,

1. Download and unpack

# Download, different packages for different operating systemsWget HTTP: / / https://artifacts.elastic.co/downloads/logstash/logstash-7.12.0-darwin-x86_64.tar.gz# decompressionThe tar - ZXVF logstash 7.12.0 - Darwin - x86_64. Tar. Gz# renameThe mv logstash - 7.12.0 logstashCopy the code

2. Logstash Command line arguments

1. Check the help information

cd logstash
bin/logstash -h
Copy the code

2. Load the specified pipeline file path

Bin /logstash -f File pathCopy the code

Note that ⚠️ : -f can be followed by a file path or a folder path. If it is a folder path, the files in that folder will be loaded to form a large pipeline file.

3. Check whether the syntax of the configuration file is incorrect

bin/logstash -t
Copy the code

4. Hot loading pipeline configuration file

bin/logstash -r
Copy the code

5. Specify the bound host and boot port

Bin /logstash --http.host 127.0.0.1 --http.port 9210Copy the code

Note ⚠️ : Refer to bin/logstash -h output for more configuration items.

3. Modify the logstash configuration file

1. Modify the JVM configuration

Vim config/jvm.options Modifies the values of other parameters such as -xms and -xmx.Copy the code

2. Modify the logstash. Yml configuration

vim config/logstash.yml

Set the name of the node
node.name: logstash-01
# 设置 pipeline 的id
pipeline.id: main
pipeline.ordered: auto
# set pipeline thread count (filter+output), default is CPU core number
# pipeline.workers: 2
# Set the location of the main pipeline file
# path.config:
Automatically reload the pipeline configuration file
config.reload.automatic: true
Set the IP and port bound to the HTTP APIHTTP. Host: 127.0.0.1 HTTP. Port: 9201Set the logstash queue type to persistent, which defaults to memory
queue.type: persisted
Copy the code

4. Write a Pipeline file

To implement the function, the interface input data from the console and then output to the console.

File name: pipeline.conf/demo-std.conf

Stdin input {stdin {codec => plain {charset => "UTF-8"} # Type => "console input" # add tag tags => ["console"]}} # filter {} # output output {stdout { codec => rubydebug { } } }Copy the code

5. Start the logstash

At startup, direct a pipeline file

# -f specifies the pipeline file path bin/logstash -f pipeline. Conf /demo-std.confCopy the code

6. View the running result

Two, the composition of pipeline

From the figure above, a pipeline consists of three elementsinput,filterandoutput. Among theminputandoutputIt’s a must.filterIt’s optional.

Input: specifies where to fetch data, such as files, Beats, TCP, and so on. Filter: we can modify the input data, such as adding fields, renaming fields, changing field types, and so on. Output: Determines where to output the processed data, such as ES, file, etc.

3. Reference links

2. Directory structure of the logstash file 3. Parameter of the logstash