Spring Boot builds ELK, this is the correct way to view logs!

I am interested in Elasticsearch, so I recorded the configuration process.

Why use ELK

ELK is actually three tools, Elastricsearch + LogStash + Kibana. Through ELK, it is used to collect logs and analyze logs. Finally, it is displayed through visual UI. At the beginning, when the service volume is small, simple SLF4J+Logger prints logs on the server, and grep is used for simple query. However, as the service volume increases, the data volume also increases, so ELK can be used to collect and analyze a large number of logs

Briefly draw the architecture diagram In the environment configuration, mainly introduces the Mac and Linux configuration, Windows system is roughly the same, of course, if you have installed JDK1.8 or later ~

[root@VM_234_23_centos ~]# java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
Copy the code

Note:

Older versions of ELK also require older JDK support, and the ELK version configured in this article is 6.0+, so a JDK version of at least 1.8 is required

ElasticSearch

Elasticsearch is a distributed, RESTful search and data analysis engine that addresses a growing variety of use cases. At the heart of the Elastic Stack, it stores your data centrally, helping you find what you expect and what you don’t expect.

Mac installation and running

Brew install ElasticSearch Run: elasticSearchCopy the code

Linux: Download the Elasticsearch file from the official address of Elasticsearch (you can also download the file using an FTP tool), decompress the gz file using tar, and go to the bin directory to run the software

[root @ VM_234_23_centos app] # curl - L - O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gz [root@VM_234_23_centos app]# tar -zxvf ElasticSearch-6.2.4.tar. gz [root@VM_234_23_centos app]# CD elasticSearch-6.2.4 [root @ VM_234_23_centos elasticsearch - 6.2.4] #. / bin/elasticsearchCopy the code

Note:

Running ElasticSearch on A Linux machine requires a new user group. At the end of this article there is a stomp record of Elastic installing on Linux

Logstash

Logstash is an open source server-side data processing pipeline that enables you to capture data from multiple sources simultaneously, transform it, and then send it to your favorite “repository.” (Our repository is, of course, Elasticsearch.) – Official cute

1. Software installation

Mac installation:

brew install logstash
Copy the code

Linux installation:

[root @ VM_234_23_centos app] # curl - L - O % of Total % at https://artifacts.elastic.co/downloads/logstash/logstash-6.3.2.tar.gz Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 137M 100 137M 0 0 5849k 0 0:00:24 0:00:24 --:--:-- 6597k [root@VM_234_23_centos app]# tar -zxvf logstuck-6.3.2.tar.gzCopy the code

2. Modify the configuration file

vim /etc/logstash.conf
Copy the code

Conf file to specify the plugin to use and configure the hosts for elasticSearch

input { stdin { } }
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}
Copy the code

3. Run

bin/logstash -f logstash.conf
Copy the code

4. Visithttp://localhost:9600/

{" host ":" = - = ", "version" : "6.2.4", "http_address" : "127.0.0.1:9600", "id" : "5b47e81f-bdf8-48fc-9537-400107a13bd2", "name": "=-=", "build_date": "2018-04-12T22:29:17Z", "build_sha": "a425a422e03087ac34ad6949f7c95ec6d27faf14", "build_snapshot": false }Copy the code

In the ElasticSearch log, you can also see the logStash log

[the 2018-08-16 T14:08:36, 436] [INFO] [O.E.C.M.M etaDataIndexTemplateService] [f2s1SD8] adding template [logstash] for the index patterns [logstash-*]Copy the code

If you see this return value, you have successfully installed and started

Hit the pit:

Java HotSpot(TM) 64-bit Server VM Warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; Error = ‘Cannot allocate memory’ (errno=12) Error = ‘Cannot allocate memory’ (errno=12) Error = ‘Cannot allocate memory’ (errno=12) As a result, logStash does not allocate enough memory, so the JVM configuration needs to be modified as a result.

[root@VM_234_23_centos logstash-6.3.2]# CD config/ [root@VM_234_23_centos config]# ll total 28 -rw-r--r-- 1 root root 1846 Jul 20 14:19 jvm.options -rw-r--r-- 1 root root 4466 Jul 20 14:19 log4j2.properties -rw-r--r-- 1 root root 8097 Jul  20 14:19 logstash.yml -rw-r--r-- 1 root root 3244 Jul 20 14:19 pipelines.yml -rw-r--r-- 1 root root 1696 Jul 20 14:19 startup.options [root@VM_234_23_centos config]# vim jvm.optionsCopy the code

Change -xms1g -xmx1g to

-Xms256m  
-Xmx256m
Copy the code

Then it will start normally

Kibana

1. Software installation

Kibana allows you to visualize Elasticsearch data and manipulate the Elastic Stack, so you can solve any questions you might have: why you’re pold at 2am, for example, or how rain affects your quarterly numbers. (And the ICONS are really cool)

Mac installation

brew install kibana
Copy the code

Linux installation, official download address

[root @ VM_234_23_centos app] # curl - L - https://artifacts.elastic.co/downloads/kibana/kibana-6.3.2-linux-x86_64.tar.gz O %  Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 195M 0 271k 0 0 19235 0 2:57:54 0:00:14 2:57:40 26393Copy the code

At this stage, the download may be extremely slow, so I download it locally and transfer it to the server through the Rz command

[root@VM_234_23_centos app]# rz rz waiting to receive. Starting zmodem transfer. Press Ctrl+C to cancel. Transferring Kibana - 6.3.2 - Linux - x86_64. Tar. Gz... Errors [root@VM_234_23_centos app]# tar -zxvf kibana-6.3.2-linux-x86_64.tar.gz [root@VM_234_23_centos app]# tar -zxvf kibana-6.3.2-linux-x86_64.tar.gzCopy the code

2. Modify the configuration

Modify the config/kibana.yml file to specify elasticSearch. url to point to elasticSearch instance.

If you use the default configuration like I did, you don’t need to modify this file

3. Start

[root@VM_234_23_centos kibana]# ./bin/kibana
Copy the code

4. Visithttp://localhost:5601/app/kibana#/home?_g=() The interface shows so much functionality that we can integrate SLF4J+LogBack

Integrating Spring + Logstash

1. Modify the logstash. Conf file and restart the logstash file

Input {# stdin {} TCP {# host:port is the destination of the appender. Enable port 9250 to receive messages from LogBack host => "127.0.0.1" port => 9250 mode => "server" tags => ["tags"] COdec => jSON_lines}} output  { elasticsearch { hosts => ["localhost:9200"] } stdout { codec => rubydebug } }Copy the code

2. Reference dependencies in Java applications

<dependency> <groupId>net.logstash.logback</groupId> <artifactId>logstash-logback-encoder</artifactId> The < version > 5.2 < / version > < / dependency >Copy the code

3. Configure log output in logback. XML

<! - log export to Logstash - > < appender name = "stash" class = "net. Logstash. Logback. Appender. LogstashTcpSocketAppender" > <destination>localhost:9250</destination> <! - the encoder must be configured with a variety of optional - > < encoder charset = "utf-8" class = "net. Logstash. Logback. Encoder. LogstashEncoder" > <! -- "appName ":"ye_test" <customFields>{" appName ":"ye_test"}</customFields> </encoder> </appender> <root level="INFO"> <appender-ref ref="stash"/> </root>Copy the code

Since I did not specify the corresponding index in the first step, the Logstash automatically created the index of Logstash -timestamp for me when the service started.

4. Add index to Kibana 5. View index information in Discover on the left 6. Add a visual diagram Visualize There are still more functions to explore, first of all, the environment will be used to continue to learn ~

Record on pit

Start the error

uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
Copy the code

Cause: The Root user cannot log in

Solution: Switch users

[root@VM_234_23_centos ~]# groupadd es [root@VM_234_23_centos ~]# useradd es -g es -p es [root@VM_234_23_centos ~]# Chown es:es /home/app/elasticSearch / # switch user to su - to get the environment variable [root@VM_234_23_centos ~]# sudo su - esCopy the code

The Exception in the thread “main” Java. Nio. File. AccessDeniedException:

Cause of error: Start ES as a non-root user and the user does not have sufficient file permissions.

Solution: chown -r User name: specifies the user name of the file or directory

For example, chown -r ABC: ABC Searchengine starts ES again

Description ElasticSearch started and reported Killed

[2018-07-13T10:19:44.775][INFO][O.E.P.luginsService][f2s1SD8] Loaded Module [aggs-matrix-stats] [2018-07-13T10:19:44.779][INFO][O.E.p.luginsService][f2s1SD8] Loaded Module [analysis-common] [2018-07-13T10:19:444,780][INFO][O.e.p.luginsService][f2s1SD8] Loaded Module [ingest-common] [2018-07-13T10:19:44.780][INFO][O.E.P.luginsService][f2s1SD8] Loaded Module [lang-expression] [2018-07-13T10:19:44.780][INFO][O.E.P.luginsService][f2s1SD8] Loaded Module [mustache] [2018-07-13T10:19:444,780][INFO][O.E.p.luginsService][f2s1SD8] Loaded Module [lang-painless] [2018-07-13T10:19:444,780][INFO][O.E.p.luginsService][f2s1SD8] Loaded Module [2018-07-13T10:19:444,780][INFO][o.e.p.pluginsService][f2s1SD8] Loaded Module [parent-join] [2018-07-13T10:19:44.780][INFO][o.e.p.luginsService][f2s1SD8] Loaded Module [percolator] [2018-07-13T10:19:44.780][INFO][o.e.p.luginsService][f2s1SD8] Loaded Module [rank-eval] [2018-07-13T10:19:444,781][INFO][O.E.p.luginsService][f2s1SD8] Loaded Module [reindex] [2018-07-13T10:19:444,781][INFO [2018-07-13T10:19:444,781][o.e.p.luginsService][f2s1SD8] loaded Module [repository-url] [2018-07-13T10:19:444,781][INFO][O.e.p.luginsService [2018-07-13T10:19:444,781][INFO][O.e.p.luginsService][f2s1SD8] loaded Module [transport-netty4] [2018-07-13T10:19:444,781 [tribe] [2018-07-13T10:19:444,781][INFO][O.e.p.luginsService][f2s1SD8] Loaded Module [X-pack-core] [2018-07-13T10:19:44.781][INFO][O.e.p.pluginsService][f2s1SD8] Loaded Module [X-pack -deprecation] [2018-07-13T10:19:44.781][INFO][o.e.p.luginsService][f2s1SD8] Loaded Module [x-pack-graph] [2018-07-13T10:19:44.781][INFO][O.E.P.luginsService][f2s1SD8] Loaded Module [X-pack-logstash] [2018-07-13T10:19:44.782][INFO][o.e.p.luginsService][f2s1SD8] Loaded Module [x-pack-ml] [2018-07-13T10:19:44.782][INFO][O.E.P.luginsService][f2s1SD8] Loaded Module [x-pack-monitoring] [2018-07-13T10:19:44.782][INFO][o.e.p.luginsService][f2s1SD8] Loaded Module [x-pack-rollup] [2018-07-13T10:19:44.782][INFO][O.E.P.luginsService][f2s1SD8] Loaded Module [x-pack-Security] [2018-07-13T10:19:44.782][INFO][o.e.p.luginsService][f2s1SD8] Loaded Module [X-pack-sql] [2018-07-13T10:19:44.782][INFO][O.E.p.luginsService][f2s1SD8] Loaded Module [x-pack-upgrade] [2018-07-13T10:19:44.782][INFO][O.e.p.luginsService][f2s1SD8] Loaded Module [x-pack-watcher] [2018-07-13T10:19:444,783][INFO][o.E.p.pluginsService][f2s1SD8] No plugins loaded on loadCopy the code

Modify jvm.options in the config directory to make the heap size smaller

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms512m
-Xmx512m
Copy the code

Insufficient virtual memory

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

[2018-07-13T14:02:06.749][DEBUG][O.E.A. ctionModule] Using REST Wrapper from plugin Org. Elasticsearch. Xpack. Security. Security [the 2018-07-13 T14:02:07, 249] [INFO] [O.E.D.D iscoveryModule] [f2s1SD8] using Discovery Type [Zen][2018-07-13T14:02:09.173][INFO][O.E.n.ode][f2s1SD8] Initialized [2018-07-13T14:02:09.174][INFO ][o.e.n.Node ] [f2s1SD8] starting ... [2018-07-13T14:02:09.539][INFO][O.E.T. ransportService][f2s1SD8] publish_address {10.105.234.23:9300}, Bound_addresses {0.0.0.0:9300} [2018-07-13T14:02:09.575][INFO][O.E.B. bootstrapchecks][f2s1SD8] bound or publishing to  a non-loopback address, enforcing bootstrap checks ERROR: [1] bootstrap checks failed [1]: max virtual memory areas vm.max_map_count [65530] is too low, Increase at least [262144] [2018-07-13T14:02:09,621][INFO][o.e.n.ode][f2s1SD8] stopping... [2018-07-13T14:02:09,726][INFO][O.E.n.ode][f2s1SD8] Stopped [2018-07-13T14:02:09,726][INFO][O.E.n.ode][f2s1SD8] closing ... [2018-07-13T14:02:09.744][INFO][O.E.n.ode][f2s1SD8] ClosedCopy the code

Need to change the size of virtual memory (under root permission)

[root@VM_234_23_centos elasticsearch]# vim /etc/sysctl.conf # insert the following code and save to exit vm. Max_map_count =655360 [root@VM_234_23_centos Elasticsearch]# sysctl -p restart elastricSearchCopy the code

Author: JingQ Source: www.sevenyuan.cn