Android Gradle Plugin, AGP for short, has long wanted to study the Android APK packaging process, after all, APK package volume optimization is one of the pre-knowledge.

However, my knowledge reserve at that time was severely insufficient, so it was really difficult to study hard. After studying Gradle for two weeks, I felt that I should have the strength to fight, so here comes this section!

More content, it is recommended to collect first, there is time to slowly fine ~

0x1. Three APK packaging flowcharts circulating on the Internet

The Android website has a new packaging flowchart (left) that is more abstract than the old one (right) and hides a lot of details:

A more detailed image was also found at the Android Studio Project Site:

If only satisfied with a write basic business Android development son, understand enough, but if you want to have deeper attainments, or suggest to learn.

For example, in an interview (my own imagination ~) :

  • Interviewer: Did YOU do APK volume optimization on your resume? Talk about what aspects of optimization have been done:
  • You: Start with the resource headers, turn PNG images into WebP, and use AndResGuard to confuse the resources;
  • Interviewer: Write a webP plugin using Gradle.
  • You: No, I usually right-click to convert directly…
  • Interviewer: Tell me the general principle of AndResGuard;
  • You:

This section takes a look at the AGP build process and a brief look at the APK packaging process

Tips: There are so many Tasks that it is impossible to read the source code one by one. There are differences between different versions of plug-ins, so it is better to teach them how to do it. The purpose of this article is to let the reader know how to find the source code when they encounter a problem.

How to view the source code of the plug-in

Research object is AGP source code, so to do a source code, there are several methods as follows:

Download the full source code

If you have enough disk space, you can download the source code of Android Gradle Plugin to the local via the repo (looks like 30 gb) :

# the latest source only 3.4.0 repo init -u https://android.googlesource.com/platform/manifest - b gradle_3. 4.0 repo syncCopy the code

2. Download some source code

Of course, if you do not need to compile, you can directly go to the corresponding source package, the following address: build-system

Click TGZ to download:

Then use a Code viewing tool like VS Code to view it

3. Take the easy way out (recommended)

Add the following dependencies to build.gradle in the app hierarchy:

Implementation 'com. Android. Tools. Build: gradle: 3.4.0'Copy the code

Under Build, go to External Libraries on the left:

0x3, some additions before reading the source code

Before reading the source code, it is recommended to review the three articles I wrote before, and add some postures:

Gradle Plugin has three types of tasks: normal Task, incremental Task and Transform.

Tasks typically inherit DefaultTask or IncrementalTask, and the @taskAction annotated method is what the Task does.

A class that inherits from IncrementalTask is an IncrementalTask. This increment is relative to full, which refers to the first compilation process after clean is called and the second compilation process after code or resources are modified. A few key approaches:

public abstract class IncrementalTask extends BaseTask {
    // Whether increments are required. The default is false
    @Internal protected boolean isIncremental(a) {}// Subclasses are required to implement tasks performed at full scale
    protected abstract void doFullTaskAction(a) throws Exception;
    
    // By default, nothing is executed, and the parameter is the file that was modified during the increment
    protected void doIncrementalTaskAction(Map<File, FileStatus> changedInputs) throws Exception{}@TaskAction
    void taskAction(IncrementalTaskInputs inputs) throws Exception {
        DoIncrementalTaskAction; otherwise doFullTaskAction
    ｝
    
    // Get the modified file
    private Map<File, FileStatus> getChangedInputs(IncrementalTaskInputs inputs) {}}Copy the code

As for Transform, Android officially provides developers with a set of APIS to modify.class files during **.class →.dex transformations. Just pay attention to the implementation of the Transform () method.

0x4. Execute Task chain for Gradle Assemble

We often package APK with the following command:

gradlew assemble
Copy the code

To see what tasks are involved in packaging once, type the following command (Linux, MAC use./gradlew) :

gradlew assemble --console=plain
Copy the code

The output results and key points are described as follows:

:app:preDebugBuild → empty task :app:preDebugBuild → empty task, Anchor: app: compileDebugAidl NO - SOURCE - > processing AIDL: app: checkDebugManifest to check whether the Manifest: app: compileDebugRenderscript NO - the SOURCE - handling renderscript: app: generateDebugBuildConfig - > generate BuildConfig. Java: app: mainApkListPersistenceDebug - generation App - list. Gson: app: generateDebugResValues - > generate resvalue, generated. XML: : app generateDebugResources task and empty, Anchor: app: mergeDebugResources - > merge resource files: app: createDebugCompatibleScreenManifests to manifest file generated in the compatible - screens, Specified screen adaptation: app: processDebugManifest to merge the manifest. The XML file: app: processDebugResources - aapt packaging resources: app: compileDebugKotlin - > Compile Kotlin file: app: prepareLintJar the UP - TO - DATE - > copy lint jar packet TO the specified location: app: generateDebugSources task and empty, Anchor: app: javaPreCompileDebug - > generate annotationProcessors json file: app: compileDebugJavaWithJavac to compile the Java file :app:compileDebugNdk → compile NDK :app:compileDebugSources → empty task, Anchor point :app:mergeDebugShaders → merge shader files :app:compileDebugShaders → compile Shaders :app:generateDebugAssets → Empty task, Anchor: app: mergeDebugAssets - > merge assests file: app: validateSigningDebug to verify the signature: app: signingConfigWriterDebug - > Write SigningConfig information: app: checkDebugDuplicateClasses - inspection repeat class: app: transformClassesWithDexBuilderForDebug - class packaged into dex App: dex transformDexArchiveWithExternalLibsDexMergerForDebug - packaged third-party libraries: app: transformDexArchiveWithDexMergerForDebug - > Packaging of the final dex: app: mergeDebugJniLibFolders - > merge jni.lib file: app: jnilibs transformNativeLibsWithMergeJniLibsForDebug - merger App: transformNativeLibsWithStripDebugSymbolForDebug - > remove the debug symbols in the native lib: app: processDebugJavaRes NO - SOURCE to deal with Java Res: app: transformResourcesWithMergeJavaResForDebug - > merge Java res: app: packageDebug - packaged apk: app: assembleDebug task and empty, Anchor: app: confused extractProguardFiles - generated file< task > < task > < task > < task > < task
Copy the code

You can also view it directly from the Build window. Double-click the Task assemble in the Gradle window on the right and look at this window:

You can also see the execution time of each Task, which is good, but not with the specific content of each Task, but with the AGP build process ~

0x5 AGP build process

As mentioned in the previous section, each Gradle plugin will be configured with a file named.properties. In this file, write the plugin implementation class and search globally to find the following file:

Open:

Point to the: AppPlugin class, and follow:

AbstractAppPlugin → BasePlugin (BasePlugin) AbstractAppPlugin → BasePlugin (BasePlugin) AbstractAppPlugin → BasePlugin

1) BasePlugin

Okay, overridden in BasePluginapply()Method, which calls two functions.basePluginApply()

Some checks are performed, followed by initialization and configuration of the plug-in, and another function, pluginSpecificApply(), is an empty implementation, followed by the process of configuring the project, configuring the extension, and creating the Tasks.

② configureProject() → configureProject

Create a DataBindingBuilder instance, force the use of no less than the minimum plug-in version currently supported, apply the Java plug-in, create a buildCache instance if the buildCache option is enabled, and add a callback to perform the reclaims-related operations after all projects have been executed.

③ configureExtension() → Configure DSL extension

Complete the following tasks:

  • Create Android DSL in Build. gradle;
  • Create instance VariantFactory, TaskManager, VariantManager;
  • ③ Register callbacks for new/removed configurations, including signingConfig, buildType, and productFlavor;
  • Create default debug signature, debug, and release buildType;

CreateTasks () → createTasks

With the createAndroidTask () :

With the createAndroidTasks () :

Note: under the traverse all the variantScope, then call createTasksForVariantData () to create variation data corresponding to the Tasks:

To: createTasksForVariantScope () :

Abstract method. To see where this method is implemented, search: extends TaskManager

Finally, the: ApplicationTaskManager class is located

CreateAnchorTasks () createAnchorTasks() createAnchorTasks()

To: createVariantPreBuildTask ()

2333, in line with the APK packaged Task chain above, the AGP plug-in build process follows here, and then learn about APK packaged Task.

0x6, Apk packaging process

Global search “XXX “, “yyy” can quickly locate the corresponding Task class, such as “compile”, “Aidl”, or search the whole Task, and then delete delete matching.

1. compileDebugAidl

Process description: convert. Aidl files into Java interface files that can be processed by the compiler using the AIDL tool: aidlcompile.java → aidlprocessor.java → call()

2. checkDebugManifest

Procedure description: check whether the androidmanifest.xml file has relevant code: checkmanifest.java

3. compileDebugRenderscript

Process description: process Renderscript files (.rs) related code: renderScriptcompile.java

4. generateDebugBuildConfig

Process description: Generate buildconfig. Java file related code: GenerateBuildConfig.java

5. mainApkListPersistenceDebug

Process description: persistent APK data to the APK – list. Gson in relevant code: MainApkListPersistence. Kt

6. generateDebugResValues

Procedure Description: Walk through the XML file in the values directory under res to generate the generated. XML code of the resValues file: GenerateResValues. Java → generate() → resValueGenerator.java

7. mergeDebugResources

Process description: use AAPT2 to merge resource files MergeResources. DoFullTaskAction () – > ResourceMerger. MergeData () – > MergedResourceWriter. The end () – > MResourceCompiler. SubmitCompile () – > AaptV2CommandBuilder. MakeCompileCommand ()

Core source code parsing:

Incremental() method has been implemented, returning true to indicate that incremental compilation is supported, following the full compilation method doFullTaskAction()

ResourcePreprocessor preprocessor = getPreprocessor();
List<ResourceSet> resourceSets = getConfiguredResourceSets(preprocessor)
Copy the code

Then go down:

Continue to:

Merger. MergeData () → Resourcemerger.mergeData () → Datamerger.mergeData ()

The MergedResourceWriter method is called with the following addItem() :

Each file creates an instance of the corresponding compileresourcerequests and adds it to the mCompileResourceRequests, which is a ConcurrentLinkedQueue, and the resource is finally processed at the end() method:

Final call AaptV2CommandBuilder. MakeCompileCommand () method to generate aapt2 command to deal with resources.

Tips: Plugins that convert images to WebP format are usually processed before this Task

8. createDebugCompatibleScreenManifests

Course description: the manifest file generated in the compatible – screens, used for screen adaptation related code: CompatibleScreensManifest. Kt

9. processDebugManifest

Process description: merge AndroidManifest. XML file related code: ProcessApplicationManifest. Java, ProcessLibraryManifest. Java

10. processDebugResources

Process description: call AAPT2 link to package resources and generate r.java file code: taskManager.java → createProcessResTask()

11. compileDebugKotlin

Compile Kotlin file as bytecode Probably in the kotlin plugin source code

12. prepareLintJar

Process description: copy lint jar packet to the specified location related code: PrepareLintJar. Java

13. avaPreCompileDebug

Process description: generate annotationProcessors. Json file related code: JavaPreCompileTask. Java

14. ompileDebugJavaWithJavac

Process description: Java file related code: AndroidJavaCompile. Java

15. compileDebugNdk

Process description: compile NDK related code: ndkcompile.java

15. mergeDebugShaders

Process description: merge Renderscript file. (rs) code: MergeSourceSetFolders. Java

16. compileDebugShaders

Process description: compile Renderscript file (.rs) related code: Shadercompile.java

17. mergeDebugAssets

Process description: merge assets file related code: MergeSourceSetFolders. Java

18. validateSigningDebug

Procedure description: verify the signature related code: ValidateSigningTask. Kt additional information: check whether the keystore file exists in the current Variant signature configuration. If the default keystore is debug keystore, the corresponding keystore will be created if it does not exist.

19. signingConfigWriterDebug

Course description: SigningConfig information related code: SigningConfigWriterTask. Kt

20. checkDebugDuplicateClasses

Process description: check the duplicate class related code: CheckDuplicateClassesTask. Kt additional information: check whether project external dependencies contains no repeat classes, packaged in dex detection error when not very friendly, so the introduction of the Task to fail fast.

21. transformClassesWithDexBuilderForDebug

Process description: will the class packaged into dex related code: DexArchiveBuilderTransform. Java

Core code parsing:

If you go to the transform() method, you can see that there are two types of class processing, the class in the directory and the class in the.jar:

With the processJarInput () :

Continue with: convertJarToDexArchive()

For both classes, we end up going to convertToDexArchive(), where launchProcessing() is called:

DexArchiveBuilder here. The convert () is called dx or d8 to play dex, along with the assignment:

22. transformDexArchiveWithExternalLibsDexMergerForDebug

Process description: third-party libraries packaging dex related code: ExternalLibsMergerTransform. Kt core code parsing:

Same as transform() :

Create a DexMergerTransformCallable instance, then the call () method:

Simple, is to adjust dx or D8 to generate the above dependencies of the library dex into a dex.

23. transformDexArchiveWithDexMergerForDebug

Process description: packaged final dex related code: DexMergerTransform. The transform () – > mergeDex () core code parsing:

With the submitForMerging () :

Also created a DexMergerTransformCallable instance, remaining logical ditto ~

24. mergeDebugJniLibFolders

Process description: merge the jni.lib file related code: MergeSourceSetFolders. Java

25. transformNativeLibsWithMergeJniLibsForDebug

Process description: merge jnilibs relevant code: MergeJavaResourcesTransform. Java

26. transformNativeLibsWithStripDebugSymbolForDebug

Process description: remove the native debug symbols related code in the lib: StripDebugSymbolTransform. Java

27. processDebugJavaRes

Process description: Java res related code: MergeJavaResourcesTransform. Java

28. transformResourcesWithMergeJavaResForDebug

Process description: merge the Java res related code: MergeJavaResourcesTransform. Java

29. packageDebug

Process description: packaged APK relevant code: PackageApplication. Java – PackageAndroidArtifact. DoTask ()

The core code is as follows:

All of the updateXxx() methods above call IncrementalPackager → updateFiles()

Finally, mApkCreator. WriteZip is called to write the above content to APK.

30. extractProguardFiles

Process description: generate confusion files related code: ExtractProguardFiles. Java

Added: Anchor Task → empty Task

The Tasks above filter the anchor Task. What is the anchor Task? A: An empty Task is used to indicate a certain state.

TaskManager → MAIN_PREBUILD: TaskManager → MAIN_PREBUILD

Along with the reference: createTasksBeforeEvaluate () :

A Task named **MAIN_PREBUILD** is registered, but no closure (Task content) is passed, that is, an empty Task.

summary

The above is all content of this section, after reading it seems to understand what, but can not say what you understand, it doesn’t matter, after all, a bit theoretical, for the back of Gradle more in-depth learning and application to do it, no need to worry, if you have any questions or in the article wrong place welcome to comment area point out, thank you ~

Hey, what time is it three o ‘clock?

References:

  • Android Gradle Plugin: Android Gradle Plugin: Android Gradle Plugin: Android Gradle Plugin

  • 【 soul 7 ask 】 deep exploration Gradle automatic building technology (five, Gradle plug-in architecture implementation principle analysis — next)

  • Gradle Learning Notes – Some tasks used during compilation

  • AAPT2

  • Android Gradle (Android gradle)