preface

Since the release of Android Studio, Gradle has been the official recommended build tool for Android. It can flexibly manage dependencies and build processes, and provides a powerful plugin system. You can easily customize plug-ins to implement customized extensions. Zhihu has introduced Android Studio and developed Gradle Plugin for a long time. This article will introduce some of zhihu’s work in this area.

practice

Similar to Android Gradle Plugin (AGP), our plug-ins are divided into application plug-in and Library plug-in, which are respectively applied in APP and Library module. As long as zhihu Gradle Plugin is used, AGP is automatically used. Versions other than the bound AGP are not allowed in principle for two reasons:

  1. The development cost of AGP compatible with multiple versions is high because AGP has no public documentation except DSL and internal API changes frequently.
  2. Internal unification to avoid incompatibility issues caused by different AGP versions, such as databinding (databinding is prone to disruptive API changes and can be uncomfortable in multi-component situations)

On language choice: The first version of the plugin was developed in Java, but found it too verbose to write compared to DSL, so I switched to Groovy. Since Groovy’s syntax is basically Compatible with Java, you can also use DSL syntax directly, with almost no cost to get started, and since it is a dynamic language, It’s very flexible to develop, comes with a lot of handy extensions, a lot of functionality is very easy to write, and can be used with @typechecked annotations to solve static checking problems, so most of the functionality is written in Groovy. As Google’s support for Kotlin has grown, a lot of AGP code has been rewritten in Kotlin, but since Groovy mixing with Kotlin is problematic and there are no obvious drawbacks to using Groovy, So for now, development is still done in Groovy.

The earliest plug-in function of Zhihu was very simple. It provided a dynamically configurable multi-channel packaging function and output it to different files. This is the function that most students used when they first came into contact with Gradle. Later, as the project became larger and larger, there were more and more developers, and various problems and requirements constantly appeared, and the functions of plug-ins were constantly enriched. Now, nearly 50 kinds of functions have been integrated. This article will select some typical functions to introduce:

Unified configuration

After componentization, there are more and more component projects, and inevitably there will be conflicts between dependency and configuration, which is not good for new students. Therefore, our plugin will add unified configuration for the project, such as:

  1. Unified compileSdkVersion, minSdkVersion, targetSdkVersion
  2. Force the use of the same support library, Kotlin, and Java versions
  3. Unify the dependent version numbers and automatically upgrade the version numbers
  4. Add Git hook
  5. Provides default configurations for unit tests, coverage, PMD, and so on
  6. Some componentization related default configurations
  7. , etc.

New developers can complete most of the general configuration by using our plugin when building a new warehouse, avoiding conflicts caused by improper configuration and reducing the complexity of new warehouse configuration. At the same time, our version number will be automatically upgraded to facilitate the unified upgrade of the general configuration.

Dependency management

Before componentization, the dependency management of single APP was very simple, and most of the codes were stored in the same warehouse, which was easy to see. However, with the deepening of componentization, there were more and more components, and problems caused by dependency gradually appeared. In addition to the limitation of unreasonable dependence mentioned in “Puzzle of Slowing down Databinding” and the multi-component combination mentioned in “Zhihu Android Client Componentation Practice”, we also added the following solutions to common problems in Plugin:

Peter Porker: Why Android DataBinding builds slowly zhuanlan.zhihu.com

icon

Peter Porker: Componentization practice of Zhihu Android Client zhuanlan.zhihu.com

icon

Limit the introduction of third-party libraries

Before componentization, Zhihu had formulated the standards for the introduction of third-party libraries. After a series of procedures, the third-party libraries were introduced in the form of merge request to the main project, which was not a problem when there was only one centralized code repository. After componentization, as soon as a dependency is added to the component, it is brought to the main project because of the transitivity of the dependency. Because the component code is scattered in different warehouses, there is no unified control of the final introduction, and the dependence of each component on the third-party library has been out of control. Many people will add their own convenient library according to their preferences, and sometimes they will introduce a third-party library because of a tiny function. Even because the existing library does not conform to their own preferences and add another situation, these often lead to function repetition, reliance on jumbled, difficult to control standardization.

Our solution is to add a global whitelist of dependencies in a remote configuration file that only a few people have write access to. The application will check if the dependencies are whitelisted at compile time. If there are non-whitelisted dependencies, the compile will report an error and direct the developer to initiate the import process. For example, zhihu uses Fresco as the image library and does not use Picasso. If you add Picasso dependency directly, you will get the following error:

image

We have eliminated the abuse of third-party libraries through mandatory whitelist checks and an introduction process

Resolving dependency conflicts

Gradle Configuration creates a dependency conflict. Gradle Configuration creates a dependency conflict with Gradle Configuration. Gradle Configuration creates a dependency conflict with Gradle Configuration

The configuration conflict

Since AGP 3.0, Android began to abandon compile to implementation and API, and officially recommended the use of implementation, however, we found in the practice process, There is no real benefit to using implementation, and if different components implement different versions, compilations are likely to report errors. Although it is possible to enforce version numbers in the app to ignore conflicts, enforcing version numbers in hundreds of libraries is a hassle. So we started recommending API instead of Implementation, but a gentleman’s agreement recommendation isn’t going to be 100 percent for everyone, so we converted implementation directly to API through plug-ins, In this way, there is no need to worry about encountering the implementation pit again. Developers also do not need to care about the difference between API and implementation. Plugin has already done everything quietly behind the scenes

Dependent content conflict

For a time, our component changes are frequent, though the version number is automatically synchronized, but does not identify the component split, merged and renamed this change, this will cause the original well, suddenly changed name or merged because of a dependency, at the same time is dependent on the old components and new components, compile time often repeated questions, If a component needs to be renamed for some reason, it has to go through all the components to change again to ensure that there is no problem with each component. The process is very troublesome. Our solution is to define a remote configuration file in which the versions and dependencies of each library are defined. The plug-in reads the configuration file when apK is compiled and automatically resolves conflicts based on the configuration file. In this way, if a component has the above changes, it can be directly connected by modifying it in the configuration file without modifying all dependent components. The dependent components can modify the dependency gradually.

No-op conflict problem

There is also an unusual dependency conflict. For example, we use Stetho in the debug version for development convenience, but we don’t need Stetho in the release version, so we use a no-op version in the release version as follows:

DebugImplementation 'com. Facebook. Stetho: stetho - okhttp3:1.5.0' releaseImplementation 'com. Zhihu. Android. Library: stetho - no - op: 1.0.0'Copy the code

But we also wanted to use stetho instead of no-op in the mrRelease version, and it was natural to think of using the mrImplementation

MrImplementation 'com. Facebook. Stetho: stetho - okhttp3:1.5.0'Copy the code

At this time, because stetho and Stetho-no-op are not the same library but there are classes with the same name, class repetition errors will be reported during compilation, so we defined a new extension to solve this problem, the new writing method is:

DebugImplementation 'com. Facebook. Stetho: stetho - okhttp3:1.5.0' releaseImplementation 'com. Zhihu. Android. Library: stetho - no - op: 1.0.0' dependencyReplace {the replace "com. Zhihu. Android. Library: stetho - no - op" with "Com. Facebook. Stetho: stetho - okhttp3:1.5.0" on "Mr"}Copy the code

This can also be used to solve the problem of having different versions of the same library for different channels

The code quality

Mandatory lint

To ensure code quality, our plugin forces Lint to run before components are released in any form to prevent seriously problematic code from being published. It also forbids ignoring Lint errors. Setting abortOnError=false in lintOptions will not work. At the same time, we will upgrade the rules found to be problematic in practice to fatal to prevent problems:

** Prevents external calls to ** Java code

Most of the code in a component should not be called externally, but Java’s weak visibility modifier makes it impossible to restrict cross-component references. Sometimes (and more habitually) we make some classes public for the sake of code structure. The code marked public can be called externally, even if the class is marked protected or the default package private. So this is often the case: There is A certain class foo. Java in component A for internal use, but the developer of component B found this class accidentally (or intentionally) when writing code and found it could be used by himself, so he directly used the method foo. bar. One day, the developer of component A changed his business. Foo. Bar () added A new parameter to the foo. bar method. It works fine by itself, but when the main project is compiled, it will find that the package fails because the B component references foo. Java.

Both Kotlin and Swift provide the internal modifier to solve this problem. Java does not have this function, but Android provides an official solution. That’s the Support-Annotations library RestrictTo, used with Lint. For example, to disable external use of foo. Java, you can do this:

@RestrictTo(RestrictTo.Scope.LIBRARY)
public class Foo {
 ...
}Copy the code

The plugin comes in handy if there are too many classes and annotating each one is a hassle. It allows you to automatically limit the RestrictTo annotation for classes, and we’ve added an @open annotation that allows only classes that are annotated to be called externally

@Open
public class Foo {
 ...
}Copy the code

Of course, if that’s not enough, the RestrictTo counterpart to the RestrictedApi, as you can see from the RestrictedApi website, is only 4/10 at Error level, and we’ll have to push it up a notch

lintOptions {
    enable "RestrictedApi"
    fatal "RestrictedApi"
}Copy the code

With mandatory Lint, this code will not be published

Resource constraints

Similar to Java code, Android resource files are normally free to be used externally. The official restriction is to declare a public. XML file and declare the resource name marked public in it. What is not declared public is private, but doing so can be tricky. Similar to the Java code constraint, our solution is: Specify a res-public folder. If a resource is added to the res-public folder, it is considered public, and other folders are considered private. Then it is clear which resources are public.

image

Also used with Lint, the corresponding rule is PrivateResource, similar to the above, not to mention.

Handwriting Parcelable is prohibited

Parcelable is an Android-specific serialization method that is officially claimed to be more efficient than Serializable, but it has a serious problem in practice: Serialization and deserialization must be in exactly the same order and type, and if they are not, some strange and difficult to troubleshoot problems can occur. Common solutions are: 1. Use IDE plug-ins to automatically generate Parcelable code; 2. Use APT annotations to automatically generate Parcelable code. But in practice it often happens like this:

  1. Some students like to show off and write Parcelable by hand
  2. Code generated using the IDE forgot to generate again when adding fields
  3. Although APT can be generated automatically, newcomers may not be aware of it or may ignore it subjectively

We will use plugin to check classes that do not automatically generate Parcelable code with APT, and report error warnings during compilation to guide developers to use automated tools

Duplicate Resource check

Different from Java code, resources of different components are allowed to have the same name. The resource name of the upper component will overwrite the resource with the same name of the lower component. When APP is running, the lower component code will find that the resource referenced is not consistent with the expected, resulting in display error or crash.

  1. For the new component, add the resource prefix. For the old components, the zhihu project is already very large when dismantling the components. Considering various dependencies, it would be too large to add prefixes altogether (although automatic tools have been provided).
  2. Compile-time checks, where our plug-in checks for conflicts between resources of the current component and resources of other components

Proguard rule restriction

In addition to obfuscating and removing useless code, proGuard also has an important function of checking for partially incompatible changes. The aforementioned modification of the foo. bar parameter can be detected by ProGurad. However, proGuard rules have a similar problem to third-party dependencies: The AAR can carry ProGuard rules with it, and in theory, developers can add any ProGuard rules they want to their components and affect the main project. Although the syntax of ProGuard is relatively simple, most students often use only soha Keep and Dontwarn:

Proguard deleted a field from proGuard

-keep class com.xxx.xxx.** { *; }Copy the code

A method reported missing, full component dontwarn a shuttle

-dontwarn com.xxx.xxx.**Copy the code

A few resourceful students will magnify their tactics

-dontwarn **Copy the code

Some crazy third party SDKS will give wild proGuard rules, and the introduced students may be undiscriminately accept the order

-ignorewarningsCopy the code

This will cause several problems. One is that blind Keep leads to some codes not being confused and compressed, resulting in unnecessary increase of package volume. Another is that blind Dontwarn leads to incompatible changes that cannot be found in time. For development, the latter has a bigger impact. So we added the ability to check proGuard rules in the plugin:

  1. Disallow the use of rules such as -*keep ***** in components. In ProGuard syntax, when representing classes, double star ****** can represent arbitrary long class names, such as com.zhihu. Will cover com.zhihu and its package under all classes and subpackages in the class, the influence scope is too large, easy to abuse, ban;
  2. Ignorewarnings disallows -ignoreWarnings. Ignorewarnings ignores all code incompatibations.
  3. Dont-dontwarn: Disallows the use of -dontwarn for a specific package. The -dontwarn command ignores all code incompatibations under the specified package. For example, -dontwarn **, which is the same as -ignoreWarnings, -dontwarn com.zhihu.** ignores all incompatibations of Zhihu codes. We have banned dontwarn wildcard rules that affect com.zhihu package code, such as -dontwarn com.** and -dontwarn com.zhihu. Library.** (which can also be extended to other packages)

If a component’s ProGuard rules violate our rules, packaging will simply hang up and bootstrap changes:

image

Automatic error-detection

Some of you may have noticed that the proGuard rule above restricts the error message at the bottom of the sample graph, which is an automatic error detection feature of our plugin. For common compilation errors, the plugin will automatically find a solution to the error until (or until) the plugin can fix it automatically. The principle is to manually collect common errors, and the error characteristics and solutions are configured in the remote end, when the compilation error when the plug-in will collect error information, and according to the error information to query whether there is a known solution, and prompt to the developer students.

In addition to the above functions, Zhihu Gradle Plugin also has many other functions, such as deleting R files, checking unreasonable resources, reducing resources, jacoco full pile insertion, etc. Many functions can be written in a separate article, but we will not expand them here

The last

To do a good job, he must sharpen his tools. Although our documentation maintenance in a team, a lot of practice, code, standard and code of conduct, but only in the document declared in these things is not enough, the practical experience tells us that depend on the self-consciousness of the rules are often unreliable, there is no guarantee that everyone be notified to you fully understand, more can’t let everyone according to the standards, Typically third-party libraries like the one mentioned above introduce restrictive features, and without strong uniform enforcement, specifications can quickly be forgotten or even bypassed. Therefore, in our continuous practice, for typical problems, while establishing the standard, we will also establish the corresponding instrument-oriented check measures, not only Gradle Plugin, but also other CI tools. Only in this way, can we better implement the standard and the specification.

At the same time, zhihu mobile platform to promote various automation and tool work is in full swing, more and more automation tools will be added in the future, if you are interested, please join Zhihu

About the author

Peter Porker, who joined Zhihu in 2016, is now the head of The Android infrastructure team of Zhihu. With rich experience in Android engineering and componentization, he designed and led the componentization split of Zhihu’s Android.