A summary,

The overall back-end architecture of the company is: Spring Boot + Dubbo. Due to the early project schedule and other reasons, there is no unified specification for the log, basically each project manages its own log. This makes it very difficult to troubleshoot later problems, especially those service scenarios that require simultaneous or multiple calls to Dubbo.

Now you need to implement log tracing from the beginning of the request to the end of the request. The requirements are simple and the implementation is not difficult. You only need to add a traceId globally.

Of course, only the log record is not enough, but also a unified log storage and query.

Second, the train of thought

2.1 Log Collection and storage

The initial choice is: Ali Cloud * log service. Landless, direct storage. The logging service supports direct sending of the Appender.

Alternative: Logback appender + message queue + ELK. However, in this case, the cost is not necessarily lower than alibaba cloud services.

2.2 Current project renovation

2.2.1 API interface

Current project returns data format:

{
    "code": 200."data": "Hello world"."msg": "ok"
}
Copy the code

After the modification, the traceId field is added to all HTTP API response bodies:

{
    "code": 200,
    "data": "Hello world"."msg": "ok"."traceId": "bd41aed8b2da4895a9d2b43d1ef12595"
}
Copy the code

2.2.2 Dubbo

For the service caller: Each time the service is invoked, traceId needs to be passed to the service provider.

For the service provider: Each time the service responds, the traceId needs to be obtained from the service caller and passed down until the response ends.

2.2.3 Log Configuration

The current log format and configuration must be unified.

2.3 Landing Roadmap

2.3.1 API interface

The project internally uses org.slf4j.MDC to pass traceId.

Use interceptors to set and clear traceId. When the request arrives, generate and set the traceId; When the request ends, the traceId is cleared.

The HTTP response body cannot be modified in the interceptor. You can write traceId to the Response Body using ControllerAdvice.

2.3.2 Dubbo

Use Dubbo org. Apache. Dubbo. RPC. The Filter to complete traceId Settings and access.

Third, note

Dubbo log problem

Dubbo service calls are not necessarily caused by HTTP requests, so some calls without traceId may occur. This one needs to be handled separately.

You can standardize the naming of traceId.

Such as:

• req:xxa:xxxRepresents front-end request, xxA module, serial number identifier XXX
• tim:xxc:xxxIndicates that the timer is triggered. The serial number of the XXC module is XXX

Naming conventions need to be written according to the specific business. Examples are for reference only

In addition, the returned data can be properly processed to avoid exposing key information and facilitate problem location and processing. Of course, this degree still needs to be grasped according to specific business and demand.