background

Code quality is a very important area in project development. Better quality code can produce fewer bugs, make developers less prone to make mistakes, and improve product quality. So how to define code quality, how to measure it, and how to present it has become the main exploration area of our internal platform Litmus.

The Litmus platform within our team has been mentioned many times in previous articles. It is an automated code review feedback system that can provide developers with analysis results of various dimensions of the current code, helping developers better solve potential bugs and better manage and organize code files.

At present, we have been iterating and running the system for nearly 4 months with good results. We have also prepared and designed subsequent open source plans. At the same time, we have optimized many aspects of ourselves from daily use. We decided to give a complete introduction to our Litmus code quality inspection platform here, how it is built and architects, as well as the thinking and summary of code quality problems generated in the construction of this platform.

What is code quality

Code quality is an abstract concept, but it permeates the entire application development process, and most bugs and delays, as well as the development experience, are caused by low code quality. We have been exploring how to define this abstract concept concretely. With the increase of business and the development of the team, the project code will become more and more difficult to maintain. These changes are not formed in a day, but a little bit of complexity and confusion brought by each submission and merger. Because there are no monitoring and manpower issues, the project will become more and more bloated and complicated. We want to step in and explore and improve the process.

First of all, we conducted a survey of the existing tools, such as sonar and some other academic inspection tools, these tools are very difficult to use, the data is slightly complex and professional, it will be an overkill in business, so we decided to build a set of quality management platform.

The syntax and style checks provided by the Lint tool are the first to come into our plan, and the easiest to see intuitive results. So we unified team TSLint ESLint and rules, the first integration in the development process, because of many members of the project and the development of habits, this step can often be avoided, so we decided to put Lint check server unified operation as a final confirmation, if there are any Lint errors will not be allowed to merge code. Secondly, we found that code repetition and code complexity are also important indicators, which are described below.

At the same time, we think the code quality should be a run automatically, the programmer should not be changed from the workflow, it should automatically analyze our code submitted, gives the result feedback, programmers accordingly modify again, need the whole process smoothly and quickly, at the same time, the results show should be as simple as possible, remove the description of specialization, Keep learning costs to a minimum and see the problem directly. With that in mind, we tried to build the Litmus platform.

Basic introduction to

Use the hooks of the internal warehouse system to touch the hooks after each project commit PR and Merge to send a request to our server. The server checks against the configuration information previously set for each project, and when the check is complete, the results are sent to the PR submitter via the internal IM tool, who can see the results on the page and possibly adjust the code accordingly.

At present, the detection mainly includes three dimensions, namely, syntax rules, copy-paste rate and code complexity:

  • Syntax rules are ESLint, TSLint, and other lint checks that count the number of errors and warnings to get a score for the code result.
  • The copy-paste rate represents the similar part in the middle of the code. A large area of similar code in the code is a place of poor quality. Problems may occur when one part is modified and another part is not modified. Calculate a reasonable score to represent the result by the number of repetitions.
  • Code complexity is a complex concept that can be explained in more detail. For this introduction, please refer to our previous detailed articles

The screenshot of the system is as follows (sensitive data is covered) :





The overall architecture

The original architecture can be found in this article, when the technology stack was very complex and didn’t scale well enough to be difficult to maintain. We continue to modify litmus, and the current architecture can be shown in this diagram:



The entire Litmus is divided into three parts, one is the Server for processing requests, this is the core; One is Core, which contains various checkers, which is a library of pure algorithms; One is MongoDB, which stores data. The entire Ltimus is written in JavaScript. Here’s a look at each component

Litmus Server

Server can be said to be the core part of the whole system, it is responsible for providing the Web interface, processing interface requests, but also accept the hook requests triggered by the warehouse system, and get data from the database to display and run the inspector. Two configuration files are required to start the Server. One is the basic information (Litmus Config) about the port of the Server and the address of the database. The other is the specific configuration information file (Project Config) for each item that needs to be checked.

Hook which is triggered when the warehouse, warehouse will initiate a request (configuration) in warehouse, request our Server, and receives the request, will know that a task needs to be checked, the task will advance check Queue (Working Queue), check the Queue is a task Queue can run multiple tasks at the same time, each time you run the task, Run the Litmus Core checker algorithm program by starting a process. Check results will be listened to via event sending and then written to the database.

Litmus Core

Core is a library of algorithms for the inspector, it’s a pure library that takes data, calculates, and produces results, not aware of any external structure, where all the inspectors are located. Note that since we are using ESLint and other Lint tools, we need to package the team’s rules as Sharable Config into an NPM package. Install this package and its peerDependencies globally on the host where the Server runs. To enable the inspector to access the correct rule file. In practice, most team ESLint rules are also a team-wide set of rules for consistency.

MongoDB

The database is used to store all the check results, and the Server can manipulate it by giving the database address in the configuration file. We have built some indexes for faster retrieval.

Litmus deployment

Early on, Litmus was designed as a project for internal team use, without considering the structure of the system for scaling out. But along with the unceasing expansion, we hope to be able to get the system operational work distribution to users of the teams, rather than a centralized management in our team, so we assume the operational mission will be a lot less, at the same time, the resources of machines secured, so I need the rest of the team can convenient compact structures, the whole system.

In order to simplify their own deployment, at the same time to other teams to test the trial version, we reference Ghost CLI developed the installation of the CLI program, can be a command to install the required various components and code. Of course, the content of the configuration file still needs to be filled out by the user.

Analysis of practical effect

In the continuous iteration, we are also paying attention to the overall use of Litmus. For this, we have selected several relatively active projects to conduct statistical analysis from the launch to the end of 2017, and made the scores into charts, showing part of the statistics here:







You can see how the quality platform is being taken seriously by the team, and how writing code is really being fed back by the Lint tools to focus on code quality. This data and our detailed usage analysis, as well as plans to improve the existing system, will be covered in detail in the next article.

Future plans

After practical use, we found that some indicators did not perform as well as we expected, and their performance was a little mediocre. Therefore, we decided to improve and upgrade the detection dimension and display mode. The current dimension is not comprehensive enough, and more indicators need to be added to better play the role of code review. While some new dimensions are being prepared and designed, they will also change the previous way of presenting data in the form of tables to the way of visualizing results in the form of structure diagrams combined with different colors. These are all explorations in our use process, and the goal is to create a comprehensive code quality measurement process that can actually lead to greater improvements for the team.

As for the open source plan that everyone is concerned about, we hope that after creating the use experience of internal teams to the utmost, we can continuously optimize ourselves through the testing experience of multiple teams inside the company, and then open source a complete set of external solutions. The open source format hasn’t been decided yet, but it’s on the agenda, and the Litmus development is still in the process of iteration.