An overview of the

Once the application is containerized, what needs to be considered is how to collect the printed logs of the application in the Docker container for operation and maintenance analysis. Typical examples are log collection for SpringBoot applications. This article describes how to use ELK logging Center to collect logs generated by containerized applications and query and analyze logs in a visual manner, as shown in the following diagram:

Note: This article was published on My public account CodeSheep. You can subscribe by holding down or scanning the heart below ↓ ↓ ↓

The mirror to

  • ElasticSearch mirror
  • Logstash mirror
  • Kibana mirror
  • Nginx images (to produce logs as containerized applications)

Enable the Rsyslog service in Linux

Modifying the Rsyslog service configuration file:

vim /etc/rsyslog.conf
Copy the code

Enable the following three parameters:

$ModLoad imtcp
$InputTCPServerRun 514

*.* @@localhost:4560
Copy the code

The intent is simple: let Rsyslog load the IMTCP module and listen on port 514, then forward the data collected in Rsyslog to local port 4560!

Then restart the Rsyslog service:

systemctl restart rsyslog
Copy the code

Check the rsyslog startup status:

netstat -tnl
Copy the code

Deploy the ElasticSearch service

docker run -d  -p 9200:9200 \
 -v ~/elasticsearch/data:/usr/share/elasticsearch/data \
 --name elasticsearch elasticsearch
Copy the code

Deploy the Logstash service

Add the following configuration file to the ~/logstash/logstash.

input {
  syslog {
    type= >"rsyslog"
    port => 4560
  }
}

output {
  elasticsearch {
    hosts => [ "elasticsearch:9200"]}}Copy the code

In the configuration we have the Logstash pull the application log data from the local Rsyslog service and forward it to the ElasticSearch database!

After the configuration is complete, you can start the Logstash container with the following command:

docker run -d -p 4560:4560 \
-v ~/logstash/logstash.conf:/etc/logstash.conf \
--link elasticsearch:elasticsearch \
--name logstash logstash \
logstash -f /etc/logstash.conf
Copy the code

Deploy the Kibana service

docker run -d -p 5601:5601 \
--link elasticsearch:elasticsearch \
-e ELASTICSEARCH_URL=http://elasticsearch:9200 \
--name kibana kibana
Copy the code

Start the nginx container to produce logs

docker run -d -p 90:80 --log-driver syslog --log-opt \
syslog-address=tcp://localhost:514 \
--log-opt tag="nginx" --name nginx nginx
Copy the code

It is obvious that the Nginx application logs in the Docker container are forwarded to the local syslog service, and then the syslog service forwards the data to Logstash for collection.

At this point, the log center has been set up and currently there are four containers in operation:

Experimental verification

  • The browser opens localhost:90 to open the Nginx interface and refreshes several times to generate a log of GET requests in the background

  • Open the Kibana visual interface: localhost:5601

  • Collect Nginx application logs

  • Querying Application Logs

Enter program=nginx in the query box to query specific logs

Afterword.

  • The author’s more original articles are here, welcome to watch

  • My Personal Blog

The author has more SpringBt practice articles here:

  • Spring Boot application monitoring actual combat
  • The SpringBoot application is deployed in an external Tomcat container
  • ElasticSearch in SpringBt practice
  • A preliminary study on Kotlin+SpringBoot joint programming
  • Spring Boot Logging framework practices
  • SpringBoot elegant coding: Lombok plus

If you are interested, take some time to read some of the author’s articles on containerization and microservitization:

  • Use K8S technology stack to create personal private cloud serial articles
  • Nginx server configuration from a detailed configuration list
  • Docker container visual monitoring center was built
  • Use ELK to build Docker containerized application log center
  • RPC framework practice: Apache Thrift
  • RPC framework practice: Google gRPC
  • Establishment of microservice call chain tracking center
  • Docker containers communicate across hosts
  • Preliminary study on Docker Swarm cluster
  • Several guidelines for writing dockerFiles efficiently