An overview of the

Lint is a Google Android static code inspection tool that scans and finds potential problems in code, notifying developers of early fixes, and improving code quality. In addition to the hundreds of Lint rules that Android provides natively, you can also develop custom Lint rules to suit your needs.

Why use Lint

During the iteration of Meituan Waimai Android App, online problems frequently occurred. It is easy to write problematic code at development time, such as the use of Serializable: classes that implement the Serializable interface Crash when serialized if objects referenced by their member variables do not implement the Serializable interface. We analyze and summarize the causes and solutions of some common problems, and share them with developers or testers to help them proactively avoid these problems.

In order to further reduce the occurrence of problems, we gradually improved some specifications, including formulating code specifications, strengthening code Review, and improving the testing process. However, these measures still have various shortcomings, including difficult implementation of code specifications, high cost of communication, especially frequent changes of developers resulting in repeated communication, so its effect is limited, similar problems still occur from time to time. On the other hand, more and more summary, standard documents, for the new members in the group also produced a lot of learning pressure.

Is there a way to reduce or mitigate these problems from a technical perspective?

Our research found that static code review is a good idea. Static code inspection frameworks, such as FindBugs, PMD, and Coverity, are available to examine Java source or class files. Checkstyle, for example, focuses on code style; But we ultimately chose to start with the Lint framework because of its many advantages:

  1. Lint can check Java source files, class files, resource files, Gradle files, etc.
  2. Extensible, supports the development of custom Lint rules.
  3. Supporting tools, Android Studio, Android Gradle plugin native support Lint tools.
  4. Lint was designed for Android and provides hundreds of useful Android-related checking rules natively.
  5. With official Support from Google, it will be updated with Android development tools.

After conducting sufficient technical research on Lint, we made some more in-depth thinking based on actual problems, including which problems should be solved with Lint and how to promote and implement Lint better, and gradually formed a set of comprehensive and effective solutions.

Introduction of Lint API

For the sake of understanding what follows, let’s take a quick look at the main APIS provided by Lint.

The main API

Lint rules are implemented by calling the Lint API, the main ones of which are as follows.

  1. Issue: represents a Lint rule.

  2. Detector: Used to detect and report on the issues in the code, each of which requires a Detector.

  3. Scope: Declares the Scope of code that the Detector will scan, such as JAVA_FILE_SCOPE, CLASS_FILE_SCOPE, RESOURCE_FILE_SCOPE, GRADLE_SCOPE, etc. An Issue can contain one to multiple scopes.

  4. Scanner: Used to scan and find issues in code, each Detector can implement one to more scanners.

  5. IssueRegistry: Entry to load Lint rules, providing a list of issues to check.

For example, the native ShowToast is an Issue, and this rule checks to see if the toast.show () call was missed after the toast.maketext () method was called. Its Detector is a ToastDetector, and the Scope to be checked is JAVA_FILE_SCOPE. ToastDetector implements JavaPsiScanner, and the schematic code is as follows.

public class ToastDetector extends Detector implements JavaPsiScanner {
    public static final Issue ISSUE = Issue.create(
            "ShowToast"."Toast created but not shown"."...",
            Category.CORRECTNESS,
            6,
            Severity.WARNING,
            new Implementation(
                    ToastDetector.class,
                    Scope.JAVA_FILE_SCOPE));
    // ...
}
Copy the code

The schematic code for IssueRegistry is shown below.

public class MyIssueRegistry extends IssueRegistry {

    @Override
    public List<Issue> getIssues(a) {
        return Arrays.asList(
                ToastDetector.ISSUE,
                LogDetector.ISSUE,
                // ...); }}Copy the code

Scanner

The main part of Lint development is implementing Scanner. Lint includes several types of Scanner, the most common of which is a Scanner that scans Java source files and XML files.

  • JavaScanner/JavaPsiScanner/UastScanner: scans Java source files
  • XmlScanner: Scans XML files
  • ClassScanner: scans class files
  • BinaryResourceScanner: Scans binary resource files
  • ResourceFolderScanner: Scans the resource folder
  • GradleScanner: Scans Gradle scripts
  • OtherFileScanner: Scans files of other types

It’s worth noting that Scanner, which scans Java source files, has gone through three versions.

  1. Javascript is used initially. Lint parses Java source code through Lombok libraries into an AST(Abstract syntax tree), which is then scanned by javascript.

  2. In Android Studio 2.2 and Lint-API 25.2.0, the Lint tool replaces Lombok AST with PSI and deprecates JavaScanner in favor of JavaPsiScanner.

    PSI is an API provided by JetBrains after parsing Java source code in IDEA to generate a syntax tree. Compared to Lombok AST, PSI supports Java 1.8, type resolution, and more. Custom Lint rules implemented using JavaPsiScanner can be loaded into Android Studio version 2.2+ and executed in real time while writing Android code.

  3. In Android Studio 3.0 and version 25.4.0 of the Lint-API, the Lint tool replaces PSI with UAST, and the new UastScanner is recommended.

    UAST is JetBrains’ API for replacing PSI in the new version of IDEA. UAST is more language-independent, supporting Kotlin in addition to Java.

This article is still introduced based on PsiJavaScanner. According to the annotations in the UastScanner source code, it is easy to migrate from PsiJavaScanner to UastScanner.

Lint rules

What problems do we need to check for in our code with Lint?

In the development process, we paid more attention to indexes such as Crash rate and Bug rate of App. Through long-term sorting and summary, it is found that there are many code problems with high frequency, whose principles and solutions are very clear, but it is easy to miss and difficult to find when writing code. Lint, on the other hand, can easily detect these problems.

Crash prevention

Crash rate is one of the most important indicators of App, and avoiding crashes has always been a headache in the development process. Lint can well detect some potential crashes. Such as:

  • Native NewApi, used to check whether code calls apis provided by higher versions of Android. Calling a higher-version API on a lower-version device causes a Crash.

  • Custom SerializableCheck. A class that implements the Serializable interface will Crash when serialized if the object referenced by its member variable does not implement the Serializable interface. We developed a code specification that requires classes that implement the Serializable interface to implement the types declared by their member variables (including those inherited from a parent class).

  • Custom ParseColorCheck. Calling color.parsecolor () to parse a background Color will result in an IllegalArgumentException that must be handled when calling the color.parsecolor () method.

Bug prevention

Some bugs can be prevented by Lint checking. Such as:

  • SpUsage: Requires all SharedPrefrence read and write operations to use the base utility class, which does various exception handling; Also define SPConstants constant class, where all SP keys will be defined to avoid conflicts between keys defined separately in the code.

  • ImageViewUsage: Check whether the ImageView has ScaleType set and whether the ImageView has Placeholder set when loading.

  • TodoCheck: Checks if there is any TODO unfinished in the code. For example, you might write some fake data in your code during development, but make sure to remove that code when you finally go live. This check item is a special one and is usually checked during the beta phase after development is complete.

Performance/security issues

Some performance and security-related issues can be analyzed using Lint. Such as:

  • ThreadConstruction: It is forbidden to create threads (except Thread pools) directly with new Threads () and requires a common utility class to perform background operations in a common Thread pool.

  • LogUsage: Do not directly use android.util.Log. Use the unified utility class. The tool class can control the Release package not to output logs, improve performance, and avoid security problems.

Code specification

In addition to code style constraints, code specifications are more about reducing or preventing bugs, crashes, performance, security, etc. While many problems are technically difficult to check directly, they are addressed indirectly by encapsulating a common base library and creating code specifications, Lint checks are used to reduce in-group communication costs, novice learning costs, and ensure code specifications are implemented. Such as:

  • SpUsage, ThreadConstruction, LogUsage, and so on mentioned earlier.

  • ResourceNaming: Resource file naming conventions to prevent resource file name conflicts between modules.

Implementation of code review

How do you alert developers to fix code problems when they are detected?

Early on we configured static code inspection on Jenkins, and when we packaged AAR/APK, we checked for problems in the code and generated reports. It turns out that while static code reviews can pick up a lot of problems, few people take the initiative to look at reports, especially when there are too many non-essential, low-priority issues (such as overly strict code style constraints).

Therefore, it is important to determine which problems to check, and when and by what technical means to perform code checks. We thought about this a little bit more in conjunction with the technical implementation and identified the main goals for static code review implementation:

  1. Focus on high-priority issues and block low-priority issues. As mentioned earlier, if a code review report contains a large number of unimportant issues, it can hinder the discovery of key issues.

  2. The solution of high – quality problems should be mandatory. When a high priority code problem is found in the inspection, the developer will be given a clear and direct error report, and the developer will be forced to fix it through technical constraints.

  3. Some problems should be discovered as soon as possible to reduce the risk or loss. Some problems are better discovered as early as possible, such as business feature development using a higher version of the Android API, which can be detected through Lint’s native NewApi. If it is found during development, other technical solutions can be considered at that time, and timely communication with product and designer when difficulties are realized; And if it comes to the code, test, or even release, online discovery, it may be too late.

Priority definition

Every Lint rule can be configured with Sevirity (priority), including Fatal, Error, Warning, Information, etc. We mainly use Error and Warning, as follows.

  • Error level: Indicates the problem that needs to be resolved, including Crash, clear Bug, serious performance problem, and code noncompliance.
  • Warning level: Includes code writing suggestions, possible bugs, and performance optimizations.

Execution time

Lint checks can be performed in multiple stages, including local manual checks, live code checks, compile-time checks, commit checks, Pull Request checks in CI systems, package release checks, etc., as described below.

Performed manually

In Android Studio, custom Lint can be run manually through the follow-inspections (dsCONFORMS Code) function.

On the Gradle command line, you can run lint checks directly with./gradlew lint.

Manual implementation is simple and easy to use, but lacks force and is easily missed by developers.

Real-time inspection of coding phase

Code-time checking is a real-time error report in the code window while writing code in Android Studio. The benefits are obvious: developers can spot code problems first. However, due to Android Studio’s imperfect support for custom Lint, the developer’s IDE is configured differently, requiring the developer to take the initiative to report errors and fix them.

IDEA provides conforms functionality and apis to implement code Inspections, which Android native Lint uses to integrate conforms to Android Studio. There doesn’t seem to be a clear official description of the custom Lint rule, but actual research has found that In Android based on JavaPsiScanner Studio 2.2 + version and development under the condition of (or Android Studio 3.0 + and JavaPsiScanner/UastScanner), the IDE will try to load and execute real-time custom Lint rules.

Technical details:

  1. In Android Studio 2.x, Menu Preferences – editor-conforms – Android-lint-frash-error from Custom Lint Check (Avaliable for Analyze | Inspect Code), according to the custom Lint only supports command line or manual operation, does not support real-time inspection.

    Error from Custom Rule When custom (third-party) lint rules are integrated in the IDE, they are not available as native IDE inspections, so the explanation text (which must be statically registered by a plugin) is not available. As a workaround, run the lint target in Gradle instead; the HTML report will include full explanations.

  2. In Android Studio 3.x, after opening the Android project source code, the IDE loads the custom Lint rules for the project, which can be found in the Conforms list of the Settings menu. It has the same effect as native Lint (Android Studio triggers a code check on the source file when it is opened).

  3. If you look at the issueregistry.getissues () method call stack for custom Lint, you can see that in Android Studio, Is by the org. Jetbrains. Android. Inspections. Lint. AndroidLintExternalAnnotator call LintDriver load perform custom lint rules.

    Reference code: https://github.com/JetBrains/android/tree/master/android/src/org/jetbrains/android/inspections/lint

In Android Studio it looks like this:

Automatic check at local compile time

Configuring Gradle scripts allows you to run Lint checks when compiling Android projects. The advantage is that problems can be found as early as possible, and there can be mandatory; The disadvantage is that the compilation speed has a certain impact.

Compiling the Android project executes the Assemble task, which allows Assemble to rely on the Lint task to perform lint checks at compile time; At the same time, LintOptions is configured to interrupt compilation if Error level problems are found.

In the Android Application Project (APK), the Android Library Project (AAR) will change applicationVariants to libraryVariants.

android.applicationVariants.all { variant ->
    variant.outputs.each { output ->
        def lintTask = tasks["lint${variant.name.capitalize()}"]
        output.assemble.dependsOn lintTask
    }
}
Copy the code

LintOptions configuration:

android.lintOptions {
	abortOnError true
}
Copy the code

Check at local commit time

With git pre-commit hook, you can perform Lint checks before committing your local code, failing which will not commit your code. The advantage of this approach is that it does not affect compilation speed at development time, but problems are found relatively late.

You can write Gradle scripts that automatically copy hook scripts from your project to the.git/hooks/ folder every time you synchronize your project.

CI check when lifting code

As part of the code submission process specification, it is a common, feasible, and effective idea to use a CI system to check Lint for problems when sending a Pull Request. Configurable CI checks must pass before code can be merged.

Jenkins is commonly used in CI systems. If Stash is used for code management, the Pull Request Notifier for Stash plug-in can be configured on Stash or the Stash Pull Request Builder plug-in can be configured on Jenkins. Implement the Job that triggers Jenkins to execute Lint checks when sending a Pull Request.

Both local compilation and code checking on CI systems can be done by executing Gradle’s Lint task. You can pass Gradle a StartParameter in CI. If the Gradle script reads this parameter, configure LintOptions to check for any Lint problems. Otherwise, only some high-priority Lint issues are checked in the local compilation environment, reducing the impact on local compilation speed.

Lint generates a report like this:

Check when packaging for release

Even if you use a CI system to run Lint checks every time you write code, there’s still no guarantee that everyone’s code will merge without a problem; In addition, for some special Lint rules, such as TodoCheck mentioned earlier, you might want to check later.

As a result, one more Lint check is required for all code before the CI system packages and releases the APK/AAR for testing or release.

Final determination of inspection timing

Considering the advantages and disadvantages of various inspection methods and our goals, we finally decided to combine the following methods to do code inspection:

  1. IDE real – time inspection during coding phase, finding problems at the first time.
  2. When compiling locally, check for high-priority problems in time and compile only after passing the check.
  3. When the code is raised, CI checks all problems and only passes the check can the code be combined.
  4. Packaging stage, complete inspection of the project, to ensure that nothing is wrong.

Configuration file Support

To facilitate code management, we created a separate project for custom Lint, which is packaged to generate an AAR to publish to the Maven repository, and the Android project being examined relies on this AAR (see the link at the end of this article).

Custom Lint, while in a standalone project, has a lot of coupling with code specifications, underlying components, and so on in the Android project being examined.

For example, we use regular expressions to check the naming conventions of resource files in the Android project. Every time the business logic changes to add the prefix of resource files, we have to modify the Lint project, publish a new AAR, and then update it to the Android project, which is very tedious. On the other hand, our Lint project will not only be used in the takeaway C Android project, but will also be used directly in other Android projects on other ends, and this varies from project to project.

So we tried to solve this problem using configuration files. For example, check the LogUsage of the Log. Different projects encapsulate different Log tool classes, and the error message should be different. Json in the module directory of the Android project being checked. The configuration files in Android Project A are:

{
	"log-usage-message": "Do not use Android.util. Log, instead use the LogUtils utility class."
}
Copy the code

The configuration file for Android Project B is:

{
	"log-usage-message": "Do not use Android.util. Log, use Logger tool class instead."
}
Copy the code

Lint gets the checked project directory from the Context object to read the configuration file, with the following key code:

import com.android.tools.lint.detector.api.Context;

public final class LintConfig {

    private LintConfig(Context context) {
        File projectDir = context.getProject().getDir();
        File configFile = new File(projectDir, "custom-lint-config.json");
        if (configFile.exists() && configFile.isFile()) {
            // Read config file...}}}Copy the code

Read configuration files, can be in the Detector beforeCheckProject, beforeCheckLibraryProject callback methods. When an error is detected in the LogUsage, an error is reported based on the information defined in the configuration file.

public class LogUsageDetector extends Detector implements Detector.JavaPsiScanner {
	// ...

	private LintConfig mLintConfig;

    @Override
    public void beforeCheckProject(@NonNull Context context) {
        // Read the configuration
        mLintConfig = new LintConfig(context);
    }

    @Override
    public void beforeCheckLibraryProject(@NonNull Context context) {
        // Read the configuration
        mLintConfig = new LintConfig(context);
    }

    @Override
    public List<String> getApplicableMethodNames(a) {
        return Arrays.asList("v"."d"."i"."w"."e"."wtf");
    }

    @Override
    public void visitMethod(JavaContext context, JavaElementVisitor visitor, PsiMethodCallExpression call, PsiMethod method) {
        if (context.getEvaluator().isMemberInClass(method, "android.util.Log")) {
        	// Get Message from the configuration file
        	String msg = mLintConfig.getConfig("log-usage-message"); context.report(ISSUE, call, context.getLocation(call.getMethodExpression()), msg); }}}Copy the code

Template Lint rules

While developing Lint rules, we found a similar set of requirements: encapsulating base utility classes that we wanted everyone to use; It’s easy for a method to throw a RuntimeException that deserves to be handled, but Java syntax doesn’t enforce handling of runtimeExceptions and often misses…

These similar requirements can also be cumbersome to develop each time in a Lint project. We tried to implement several templates that allow you to configure Lint rules directly from a configuration file in the Android project.

The following is an example configuration file:

{
  "lint-rules": {
    "deprecated-api": [{
      "method-regex": "android\\.content\\.Intent\\.get(IntExtra|StringExtra|BooleanExtra|LongExtra|LongArrayExtra|StringArrayListExtra|Serial izableExtra|ParcelableArrayListExtra).*"."message": "Avoid calling intent.getxx () with IntentUtils because crashes may occur on specific models."."severity": "error"
    },
    {
      "field": "java.lang.System.out"."message": "Do not use system. out directly, use LogUtils instead"."severity": "error"
    },
    {
      "construction": "java.lang.Thread"."message": "Avoid creating separate threads to perform background tasks because of performance problems. AsyncTask is recommended."."severity": "warning"
    },
    {
      "super-class": "android.widget.BaseAdapter"."message": "Avoid using a BaseAdapter directly and use a unified wrapped BaseListAdapter instead."."severity": "warning"}]."handle-exception": [{
      "method": "android.graphics.Color.parseColor"."exception": "java.lang.IllegalArgumentException"."message": "Color.parseColor requires a try-catch to handle IllegalArgumentException"."severity": "error"}}}]Copy the code

Two types of template rules are defined in the sample configuration:

  • DeprecatedApi: Disallows direct calls to the specified API
  • HandleException: When calling the specified API, a try-catch is added to handle the exception of the specified type

Problem API matching, including method calls (method), member variable references (field), constructors (construction), inheritance (super-class), and other types; Match strings support glob syntax or regular expressions (the same configuration syntax as ignore in lint.xml).

Implementation, mainly through Java syntax tree translates into specific types of nodes in the complete string (such as method calls android. The content. Intent. GetIntExtra), then check if there is a template rule matching. After the match is successful, the DeprecatedApi rule directly outputs message indicating an error. The HandleException rule checks to see if the matched node handled a particular Exception (or an Exception’s parent) and reports an error if it did not.

Check the new files by Git version

As new rules were being developed for Lint, we ran into another problem. There is a large amount of historical code in the Android project that does not meet the requirements of the new Lint rule, but does not cause significant problems. In this case, adding the new Lint rule requires all historical code to be modified, which is costly and risky. For example, the new code specification requires the use of a common thread utility class instead of allowing direct use of handlers to avoid memory leaks.

We tried a compromise: only check files added after specifying git commit. Add a configuration item to the configuration file, and configure the Git-base attribute for the Lint rule with the value of commit ID. Only files added after the commit are checked.

To achieve this, run git rev-parse –show-toplevel to obtain the root directory of the Git project. Run git ls-tree –full-tree –full-name –name-only -r

to obtain the list of existing files (relative to the git root directory) at the specified commit time. In the Scanner callback method, use context.getLocation (node).getFile() to get the file where the node is located and determine whether to check the node based on the git file list. It is important to consider the performance cost of Lint checking on your computer for large amounts of code.

conclusion

Over time, Lint static code checking has been found to work very well for specific problems, such as finding low-level errors that are explicit at the language or API level and helping enforce code specification constraints. Before Lint, many of these issues happened to be easy for developers to miss (e.g. the native NewApi check, the custom SerializableCheck); The same problems recurred; The implementation of code specifications, especially when new people are involved in development, requires high learning and communication costs, and it is often the case that new people are repeatedly asked to change their code because they do not comply with the code specification. After using Lint, these problems can be solved in the first time, saving a lot of manpower, improving code quality and development efficiency, and improving the experience of using the App.

Resources and extended reading

References:

  • Use Lint to improve your code | Android Studio
  • Android Plugin DSL Reference: LintOptions
  • Android custom Lint practices
  • Source code Analysis for the Lint tool (3)
  • Android Studio Release Notes
  • Git – Documentation

Lint and Gradle technical details can also be found on my personal blog:

  • Android Lint: Basic usage and configuration
  • Android Lint: Custom Lint debugging and development
  • Android Gradle Configuration quick start
  • Gradle Development Quick Start – DSL syntax principles and common API introduction

Author’s brief introduction

Zi Jian, senior Engineer of Android, graduated from Xidian University in 2015 and joined Meituan. I have been responsible for the development and maintenance of the home page, merchant container, evaluation and other core business modules of takeout App in the early stage. At present, I am mainly responsible for the technical work of takeout packaging automation, code checking, platformization and so on.

recruitment

If you are interested in our team, you can follow our column. Meituan takeout App team is looking for senior Android/iOS engineers/technical experts, working in Beijing/Shanghai is optional, welcome interested students to send resumes to wukai05#meituan.com.