One, foreword
The project of the company has made a cluster to realize the request diversion. Since there are more or less online request failures or system anomalies, in order to check the log information of failed requests, we have to open the log files of all services to locate and analyze the problems, which is very troublesome to operate. Therefore, our development team decided to design a log viewing system to solve the above problems.
Second, implementation ideas
By default, application service logs are stored in a directory on the local server. To view the logs, you should output the logs of multiple servers to one log file.
Due to the Logback logging framework and RabbitMQ message queues used by the project, the two can be integrated.
So we can output the logs from the project code to the RabbitMQ queue, read the queue data through Logstash, and finally to a log file.
Three, prepare the environment
Test environment: CentOS 7 whose IP address is 192.168.2.13
# 3.1 RabbitMQ configuration
To set up the RabbitMQ environment, see CentOS 7.2 Installing RabbitMQ in this website.
After the RabbitMQ configuration is complete, log in to the RabbitMQ management page and perform the following steps:
- Create a queue named log_queue
- Create a switch named Rabbit.log (direct type)
- Bind the log_queue queue to the Rabbit.log switch
Operation diagram:
# 3.2 Logstash configuration file
Modify the Logstash configuration file. There are also blog posts about Logstash on this site. You can check them out in The Basics of Logstash.
input { rabbitmq { type =>"all" durable => true exchange => "rabbit.log" exchange_type => "direct" key => "info" host => "192.168.2.13" port => 5672 user => "light" password => "light" queue => log_queue" auto_delete => false}} output { file { path => "/usr/test-log/test-%{+YYYY-MM-dd}.log" codec => multiline { pattern => "^\d" negate => true what => "previous" } } }Copy the code
Note: Multiline is a Logstash plug-in that needs to be installed manually.
The configuration indicates that the Logstash service reads logs from RabbitMQ and outputs them to a specified directory file.
Four, coding
# 4.1 rely on
List the main dependencies:
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-amqp</artifactId> </dependency> <dependency> <groupId>org.springframework.amqp</groupId> <artifactId>spring-rabbit</artifactId> </dependency>Copy the code
Log files
Called logback – spring. XML
<? The XML version = "1.0" encoding = "utf-8"? > <configuration debug="false"> <! <property name="LOG_HOME" value="d:/" /> <! - the console output - > < appender name = "STDOUT" class = "ch. Qos. Logback. Core. ConsoleAppender" > < encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <! -- Format output,%d: date; %thread: specifies the thread name. %-5level: level, 5 characters wide from the left; % MSG: log message; %n: newline -> <pattern>%d{YYYY-MM-DD HH: MM :ss.SSS} [%thread] %-5level %logger{50} - % MSG %n</pattern> </appender> <appender name="RABBITMQ" class="org.springframework.amqp.rabbit.logback.AmqpAppender"> <layout> <pattern><! [CDATA[ %d %p %t [%c] - <%m>%n ]]></pattern> </layout> <! -- rabbitMQ address --> <addresses>192.168.2.13:5672</addresses> <username>light</username> <password>light</password> <declareExchange>true</declareExchange> <exchangeType>direct</exchangeType> <exchangeName>rabbit.log</exchangeName> <routingKeyPattern>info</routingKeyPattern> <generateId>true</generateId> <charset>UTF-8</charset> <durable>true</durable> <deliveryMode>NON_PERSISTENT</deliveryMode> <autoDelete>false</autoDelete> </appender> <logger name="com.light.rabbitmq" level="info" additivity="false"> <appender-ref ref="STDOUT"/> <appender-ref ref="RABBITMQ"/> </logger> <! -- Log output level, level default DEBUG, root is logger, It is logger root --> <root level="INFO"> <appender-ref ref="STDOUT" /> <appender-ref ref="RABBITMQ" /> </root> </configuration>Copy the code
ExchangeType and exchangeName in the configuration are the type and name of the switch we created above.
# 4.3 Test classes
Custom exceptions:
public class CustomException extends RuntimeException{ private static final long serialVersionUID = 1L; private int code; private String msg; public CustomException(int code, String msg) { super(msg); this.code = code; this.msg = msg; } public int getCode() { return code; } public void setCode(int code) { this.code = code; } public String getMsg() { return msg; } public void setMsg(String msg) { this.msg = msg; }}Copy the code
Simulated log printing:
@Component public class DemoTask { private static Logger logger = LoggerFactory.getLogger(DemoTask.class); private int num = 1; @Scheduled(fixedRate = 3000) public void writeLog() { try { if (num % 5 == 0) { throw new CustomException(500, "Custom exception error "); } logger.info("==={}===={}","hello rabbitmq", System.currentTimeMillis()); num++; } catch (CustomException e) { e.printStackTrace(); logger.error("=={}==", e); }}}Copy the code
5. Code testing
Execution startup class:
@EnableScheduling
@SpringBootApplication
public class RabbitmqTestApplication {
public static void main(String[] args) {
SpringApplication.run(RabbitmqTestApplication.class, args);
}
}
Copy the code
The execution result is shown as follows:
Log information about code execution has been printed to the specified log file.
supplement
Since the logs of multiple servers are printed to the same file, you need to print the HOST IP address corresponding to the log information to distinguish the log source. The specific implementation steps are as follows:
- Custom log converter
You need to inherit from the ClassicConverter class
public class CustomLogConverter extends ClassicConverter { public String convert(ILoggingEvent event) { try { return InetAddress.getLocalHost().getHostAddress(); } catch (UnknownHostException e) { e.printStackTrace(); } return null; }}Copy the code
- Modify the logback-spring. XML file
Only key configuration information is displayed
<conversionRule conversionWord="ip" converterClass="com.light.rabbitmq.log.CustomLogConverter" /> <appender name="RABBITMQ" class="org.springframework.amqp.rabbit.logback.AmqpAppender"> <layout> <pattern><! [CDATA[%ip %date{yyyy-MM-dd HH:mm:ss} | %highlight(%-5level) | %yellow(%thread) | %green(%logger) | %msg%n ]]></pattern> </layout> </appender>Copy the code