What is Cat?

Central Application Tracking (CAT) is a real-time Application monitoring platform developed based on Java, including real-time Application monitoring and service monitoring.

As the basic component of the server project, CAT provides Java, C/C++, Node.js, Python, Go and other multi-language clients, as well as the infrastructure middleware framework (MVC framework, RPC framework, database framework, cache framework, message queue, etc.) reviewed in Meituan. Deep integration of meituan Dianping business lines to provide rich system performance indicators, health status, real-time alarms, etc.

The great advantage of CAT is that it is a real-time system. Most of CAT systems are minute-level statistics, but the second level is defined as 48 minutes and 40 seconds from data generation to the end of server processing. Basically, we can see 48 minutes and 38 seconds of data. Second advantage, monitoring data is full statistics, the client is expected to calculate; Link data is a sampling calculation.

Github: https://github.com/dianping/cat[1]

Cat Highlights

  • Real-time processing: The value of information diminishes over time, especially during incident processing
  • Full data: Collects full indicator data for in-depth analysis of fault cases
  • High availability: Fault recovery and fault location require high availability monitoring
  • Fault tolerance: A fault does not affect the normal running of services and is transparent to services
  • High throughput: The collection of massive monitoring data requires high throughput capacity
  • Scalable: supports distributed, cross-IDC deployment and horizontal expansion of the monitoring system

Why Cat?

Scenario 1: Users give feedback that the App cannot place orders, the payment cannot be made, and the products cannot be searched

The problem with scenario 1 is that when something goes wrong with the system, the user is always the first to respond. What we need to do is to be the first developer to know when something is wrong, not the user to tell us.

Cat provides a second-level alarm mechanism when a fault occurs, instead of waiting for users to report the fault.

Scenario 2: How to quickly locate a fault

The traditional way is when something goes wrong, we go to the server and see if the service is still alive. If yes, the log is checked to see if there is any exception information.

On the home page of Cat background, the running status of each system will be displayed. If there is any abnormality, a large area of red will be displayed, which is very obvious. The most direct way is to directly view the Problem report, which displays direct exception information to quickly locate problems.

The picture

Scenario 3: User feedback The order list takes 10 seconds to display, and the user feedback that the order keeps going around and around

Scenario 3 is related to optimization. For R&D, optimization is a long-term process, and there is no best but better. Optimization in addition to the need to have a corresponding program, the most important is to the right medicine.

The idea is that before you optimize, you need to know what’s slow. Slow RPC calls? Slow database query? Slow cache updates?

Cat can provide detailed performance data, 95 lines, 99 lines, etc. More granular is the ability to see all the time-consuming logic for a request or a business method, if you do a buried operation.

The Cat statements

Cat currently has five types of reports, each of which has a specific application scenario. Let’s take a closer look at the use of these reports.

The Transaction report

Suitable for monitoring a piece of code, such as: run times, QPS, error times, failure rate, response time statistics (average impact time, Tp quantile value), and so on.

Embedding method:

public void shopService() { Transaction transaction = Cat.newTransaction("ShopService", "Service"); try { service(); transaction.setStatus(Transaction.SUCCESS); } catch (Exception e) { transaction.setStatus(e); Cat.logerror (e); cat.logerror (e); // The exception is reported to cat // optionally thrown up: throw e; } finally { transaction.complete(); }}Copy the code

Rpc, database, and other frameworks can be buried in the base framework so that these components can be monitored through Cat.

Cat Transaction can also be used in businesses that need burial points, such as placing orders, paying and other core functions. Usually, we can bury URLS, which contain specific business processes.

The picture

The Event report

It is useful for monitoring the number of times a piece of code is run, such as how many times an event is recorded and how many times an error occurred in the program. The overall structure of the Event report is almost the same as that of the Transaction report, except that the response time statistics are missing.

Embedding method:

 Cat.logEvent("Func", "Func1");
Copy the code
The picture

The Problem statement

Problem Records problems that occur during the running of the entire project, including exceptions, errors, and long access behaviors.

If someone feedback that your interface is reporting error 500, you can directly go to the Problem report after entering Cat, and the error message is in the Problem.

The picture

Problem reports do not need to be buried manually. We can simply integrate the LogAppender for logs into our project to log all error exceptions. The following sections explain how to do this.

The Heartbeat report

The Heartbeat report is a CAT client that periodically reports the status of the current running time to the server on a one-minute basis.

The picture

System indicators Include system load information, memory usage, and disk usage.

JVM metrics have GC related information, thread related information.

The Business report

Business reports correspond to Business metrics, such as order metrics. Different from Transaction, Event, and Problem, Business is more focused on macro indicators, while the other three are more focused on micro code execution.

I have not used this report very much, and most of them are the first few.

The picture

Cat in Kitty Cloud

The basic component of Kitty Cloud is Kitty, which has a layer of packaging for the required frames, such as expansion, adding Cat burying points and other functions.

The Cat integration

Kitty has a layer of Cat package, which can be integrated into the project by directly relying on kitty-Spring-cloud-starter-CAT.

 <dependency>
       <groupId>com.cxytiandi</groupId>
       <artifactId>kitty-spring-cloud-starter-cat</artifactId>
       <version>Kitty Version</version>
 </dependency>
Copy the code

Then configure the server address information of Cat in the application configuration file, separated by multiple English commas:

The servers = 47.105.66.210Copy the code

Create a meta-INF directory in the resources directory of your project and configure app.name by creating the app.properties file in the meta-INF directory. This name is the application name displayed in the Cat background

app.name=kitty-cloud-comment-provider
Copy the code

Finally, you need to configure the LogAppender of Cat so that when the application logs error level logs, Cat can perform alarm operations in time.

Add the following configuration to logback.xml:

 <appender name="CatAppender" class="com.dianping.cat.logback.CatLogbackAppender"></appender>
 <root level="INFO">
     <appender-ref ref="CatAppender" />
 </root>
Copy the code

For more details, please visit Cat’s Github page.

MVC framework buried point

For Web application development based on Spring Boot, the most commonly used Starter package is spring-boot-starter-Web.

If you use Kitty to build a microservice framework, you no longer need to rely directly on spring-boot-starter-Web. Instead, it relies on kitty-spring-cloud-starter-Web in Kitty.

Kitty-spring-cloud-starter-web encapsulates on the basis of spring-boot-starter-Web. It will perform Cat burying on the requested Url and receive and pass through some general information. Calls to RestTemplate are cat-buried.

Rely on kitty-spring-cloud-starter-web in the project:

<dependency>
      <groupId>com.cxytiandi</groupId>
      <artifactId>kitty-spring-cloud-starter-web</artifactId>
      <version>Kitty Version</version>
</dependency>
Copy the code

Start the project, then access your REST API. You can see the URL monitoring information in the Cat console.

The picture

Click on the URL to see the specific URL information.

The picture

Further, you can see the entire URL information, such as database queries, cached operations, Http calls, and so on. When the back-end students optimize the performance, they can directly start from the URL to analyze the link time of the entire request.

The picture

Mybatis buried point

In Kitty, Mybatis is Mybatis Plus, which is mainly used for Cat burying SQL related to database operations, so that it is convenient to check the time consumption of SQL.

Rely on Kitty – spring – the cloud – starter – mybatis:

<dependency> <groupId>com.cxytiandi</groupId> <artifactId>kitty-spring-cloud-starter-mybatis</artifactId> <version>Kitty  Version</version> </dependency>Copy the code

Other use methods are the same as Mybatis Plus, please refer to Mybatis Plus documentation: https://mp.baomidou.com[2]

As long as the database operations are involved, the data will be displayed in Cat.

The picture

Click SQL and you can also see which Mapper is operating.

The picture

Further, you can see the specific SQL statements and the elapsed time.

The picture

With this data, backend developers can optimize their SQL.

Redis buried point

If you need to use Spring Data Redis, you can integrate kitty-spring-cloud-starter-redis directly. In kitty-spring-cloud-starter-redis, redis commands are buried, and the corresponding commands and elapsed time can be viewed intuitively on Cat.

Add the corresponding Maven dependency:

<dependency>
     <groupId>com.cxytiandi</groupId>
     <artifactId>kitty-spring-cloud-starter-redis</artifactId>
     <version>Kitty Version</version>
 </dependency>
Copy the code

Use StringRedisTemplate directly:

@Autowired
private StringRedisTemplate stringRedisTemplate;
 
stringRedisTemplate.opsForValue().set("name", "yinjihuan");
Copy the code

You can see Redis information in Cat.

The picture

Click on Redis to see what commands are available.

The picture

You can see the details of the command, such as the key of the operation and the elapsed time.

The picture

Directing a buried point

In Kitty, Spring Data Mongodb is encapsulated, and only MongoTemplate is buried. You need to rely on kitty-spring-cloud-starter-mongodb.

<dependency> <groupId>com.cxytiandi</groupId> <artifactId>kitty-spring-cloud-starter-mongodb</artifactId> <version>Kitty  Version</version> </dependency>Copy the code

After the Mongo operation occurs, the relevant data can be seen on Cat.

The picture

Click in to see which method of MongoTemplate is invoked.

The picture

Further, you can see the specific Mongo parameters and elapsed time.

The picture

Dubbo, Feign, Jetcache, ElasticSearch, etc.

Cat uses a little trick

Buried point utility class

If we want to monitor business methods, we usually use Transaction, which includes the business logic in the Transaction to monitor the time consuming information of the business.

The method of burying points is also carried out through cat.newTransaction. For details, refer to the burying points listed in the Transaction introduction above.

The best way to do this is to have a unified tool class to encapsulate the details of the burying point.

public class CatTransactionManager { public static <T> T newTransaction(Supplier<T> function, String type, String name, Map<String, Object> data) { Transaction transaction = Cat.newTransaction(type, name); if (data ! = null && ! data.isEmpty()) { data.forEach(transaction::addData); } try { T result = function.get(); transaction.setStatus(Message.SUCCESS); return result; } catch (Exception e) { Cat.logError(e); if (e.getMessage() ! = null) { Cat.logEvent(type + "_" + name + "_Error", e.getMessage()); } transaction.setStatus(e); throw e; } finally { transaction.complete(); }}}Copy the code

Tool class use:

public SearchResponse search(SearchRequest searchRequest, RequestOptions options) {
    Map<String, Object> catData = new HashMap<>(1);
    catData.put(ElasticSearchConstant.SEARCH_REQUEST, searchRequest.toString());
    return CatTransactionManager.newTransaction(() -> {
        try {
            return restHighLevelClient.search(searchRequest, options);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }, ElasticSearchConstant.ES_CAT_TYPE, ElasticSearchConstant.SEARCH, catData);
}
Copy the code

By using utility classes, you no longer need to set up whether a Transaction is complete or successful in every monitoring place.

Annotations buried point

To make Transaction easier to use, we can customize annotations to do this. For example, if you need to monitor core business methods such as ordering, payment, etc., you can use custom Transaction annotations to add to the methods, and then use AOP to do unified monitoring.

Define notes:

@Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, Elementtype.type}) public @interface CatTransaction {/** * TYPE, default Method * @return */ String TYPE () default ""; /** * class name by default. * @return */ String name() default ""; /** * Whether to save parameter information to Cat * @return */ Boolean isSaveParamToCat() default true; }Copy the code

Define section:

@Aspect public class CatTransactionAspect { @Around("@annotation(catTransaction)") public Object aroundAdvice(ProceedingJoinPoint joinpoint, CatTransaction catTransaction) throws Throwable { String type = catTransaction.type(); if (StringUtils.isEmpty(type)){ type = CatConstantsExt.METHOD; } String name = catTransaction.name(); if (StringUtils.isEmpty(name)){ name = joinpoint.getSignature().getDeclaringType().getSimpleName() + "." + joinpoint.getSignature().getName(); } Map<String, Object> data = new HashMap<>(1); if (catTransaction.isSaveParamToCat()) { Object[] args = joinpoint.getArgs(); if (args ! = null) { data.put("params", JsonUtils.toJson(args)); } } return CatTransactionManager.newTransaction(() -> { try { return joinpoint.proceed(); } catch (Throwable throwable) { throw new RuntimeException(throwable); } }, type, name, data); }Copy the code
}

Annotations use:

@CatTransaction
@Override
public Page<ArticleIndexBO> searchArticleIndex(ArticleIndexSearchParam param) {
}
Copy the code

A few questions you might be concerned about

Can Cat do link tracking?

Cat is mainly a real-time monitoring system, not a standard full link system, mainly Cat logView is not suitable for asynchronous threads and some other scenarios, Cat model is not suitable for this. On Github of Cat, there is a mTrace dedicated to full-link analysis in Meituan Dianping.

But if seamless data transfer is done in Mvc, remote call and other frameworks, Cat can also act as a link tracking system, the basic scenario is enough.

Cat can also build a remote message tree, where you can see which services the request went through and how long each service took. But the dependency diagram between services is not in Cat.

The following request is forwarded from the gateway to Articles, which then invokes the Users interface.

The picture

Cat or Skywalking?

Skywalking is also an excellent APM framework that I haven’t used yet, but I’ve seen some documentation, and it’s full of features and a nice interface. The biggest advantage is that applications are not monitored in a bytecode-enhanced way, as is the case with Cat.

The reason for this subheading is because if you haven’t used it yet you’re going to struggle with which one to monitor. I personally think both of these are ok, you can make a simple version of the experience experience, combined with the function points you want to evaluate which landing.

The best way to use Cat is to have a basic framework, buried in the basic framework, so that Cat can display detailed information to help us quickly locate problems and optimize performance.

Interested in the Star of bai: https://github.com/yinjihuan/kitty-cloud [3]

About the author: Yin Jihuan, a simple technology lover, the author of Spring Cloud Micro-service – Full stack Technology and Case Analysis, Spring Cloud Micro-service Introduction combat and Progress, the founder of the public account Monkey World. Personal wechat jihuan900, welcome to hook up.

The resources

[1]

https://github.com/dianping/cat: https://github.com/dianping/cat


[2]

https://mp.baomidou.com: https://mp.baomidou.com


[3]

https://github.com/yinjihuan/kitty-cloud: https://github.com/yinjihuan/kitty-cloud

If you are interested, you can pay attention to my wechat public number, Simtiandi, and read more technical articles for the first time. I also have some open source code on GitHub github.com/yinjihuan