preface
Previous article “Always hear about AGP, what does it actually do?” We analyzed what AGP(Android Gradle Plugin) does, and learned that AGP is for the packaging process.
So, this article will talk to you about the Transform of them with why the AGP 3. 7.0.x.x version can be obtained by reflection transformClassesWithDexBuilderForXXX Task in 4.0.0 version will work?
Source code go!
I. Process of Transform
Before reading this article, I believe that students already have the basic knowledge of Transform.
I’m sure many of you have seen this picture:
As shown in the image above, we can see:
- In a project, we might have both custom
TransformAnd there will be systemsTransform - During processing, each Transform receives the output stream of the previous Transform, and the original file stream goes through many transforms
2. Transform source code analysis
Now that we understand the overall process, let’s look at the details.
The starting point of the first Transform
We all know that the purpose of a Transform is to modify the bytecode, so where do these Class files come from?
AbstractAppTaskManager AGP < AbstractAppTaskManager > < AbstractAppTaskManager > < AbstractAppTaskManager > < AbstractAppTaskManager >
private void createCompileTask(@NonNull VariantPropertiesImpl variantProperties) {
ApkCreationConfig apkCreationConfig = (ApkCreationConfig) variantProperties;
/ / execution javac
TaskProvider<? extends JavaCompile> javacTask = createJavacTask(variantProperties);
// Add the Class input stream
addJavacClassesStream(variantProperties);
setJavaCompilerTask(javacTask, variantProperties);
// Perform transform and dex related tasks
createPostCompilationTasks(apkCreationConfig);
}
Copy the codeThere are only a few methods, but each method can be quite powerful. Let’s start with Javac.
The second step is to execute javac
You’ll be familiar with the Javac command that converts.java files into.class files. And this is true:
public TaskProvider<? extends JavaCompile> createJavacTask(
@NonNull ComponentPropertiesImpl componentProperties) {
// Java precompile task, mainly processing Java annotations
taskFactory.register(new JavaPreCompileTask.CreationAction(componentProperties));
// Java compilation task
final TaskProvider<? extends JavaCompile> javacTask =
taskFactory.register(new JavaCompileCreationAction(componentProperties));
postJavacCreation(componentProperties);
return javacTask;
}
Copy the codeIts method comments:
Creates the task for creating *.class files using javac. These tasks are created regardless of whether Jack is used or not, but assemble will not depend on them if it is. They are always used when running unit tests.
Obviously, to create a.class file.
The most important step in this process is to register a Task called JavaCompile, which converts Java files and Java annotations into a.class Task.
JavaCompiler () : JavaCompiler () : JavaCompiler ()
Of course, not only.class files, but others such as.kt and.jar require special tasks to be converted into the input sources we need.
The third step is to create the original input stream
Back to the first step, go to the addJavacClassesStream method:
protected void addJavacClassesStream(@NonNull ComponentPropertiesImpl componentProperties) {
// create separate streams for the output of JAVAC and for the pre/post javac
// bytecode hooks
TransformManager transformManager = componentProperties.getTransformManager();
boolean needsJavaResStreams =
componentProperties.getVariantScope().getNeedsJavaResStreams();
transformManager.addStream(
OriginalStream.builder(project, "javac-output")
// Need both classes and resources because some annotation
// processors generate resources
.addContentTypes(
needsJavaResStreams
? TransformManager.CONTENT_JARS
: ImmutableSet.of(DefaultContentType.CLASSES))
.addScope(Scope.PROJECT)
.setFileCollection(project.getLayout().files(javaOutputs))
.build());
BaseVariantData variantData = componentProperties.getVariantData();
transformManager.addStream(
OriginalStream.builder(project, "pre-javac-generated-bytecode")
.addContentTypes(
needsJavaResStreams
? TransformManager.CONTENT_JARS
: ImmutableSet.of(DefaultContentType.CLASSES))
.addScope(Scope.PROJECT)
.setFileCollection(variantData.getAllPreJavacGeneratedBytecode())
.build());
transformManager.addStream(
OriginalStream.builder(project, "post-javac-generated-bytecode")
.addContentTypes(
needsJavaResStreams
? TransformManager.CONTENT_JARS
: ImmutableSet.of(DefaultContentType.CLASSES))
.addScope(Scope.PROJECT)
.setFileCollection(variantData.getAllPostJavacGeneratedBytecode())
.build());
}
Copy the codeThe transformManager handles the Transform and is building the raw data stream of the first Transform.
Attentive students may be found, the first data flow is DefaultContentType contentType. The CLASSES, the scope is the scope. The PROJECT, the custom of the Transform students must know, This sets up our custom Transform to receive the raw data stream.
Step 4 Create the compiled task
Back to the first step of createPostCompilationTasks method, it is used to create the compiled task:
public void createPostCompilationTasks(@NonNull ApkCreationConfig creationConfig) {
/ /...
TransformManager transformManager = componentProperties.getTransformManager();
// ...
/ / java8 sugar
maybeCreateDesugarTask(
componentProperties,
componentProperties.getMinSdkVersion(),
transformManager,
isTestCoverageEnabled);
BaseExtension extension = componentProperties.getGlobalScope().getExtension();
// Merge Java Resources.
createMergeJavaResTask(componentProperties);
// ----- External Transforms -----
// apply all the external transforms.
List<Transform> customTransforms = extension.getTransforms();
List<List<Object>> customTransformsDependencies = extension.getTransformsDependencies();
boolean registeredExternalTransform = false;
for (int i = 0, count = customTransforms.size(); i < count; i++) {
Transform transform = customTransforms.get(i);
List<Object> deps = customTransformsDependencies.get(i);
registeredExternalTransform |=
transformManager
.addTransform(
taskFactory,
componentProperties,
transform,
null,
task -> {
if(! deps.isEmpty()) { task.dependsOn(deps); } }, taskProvider -> {// if the task is a no-op then we make assemble task depend on it.
if (transform.getScopes().isEmpty()) {
TaskFactoryUtils.dependsOn(
componentProperties
.getTaskContainer()
.getAssembleTask(),
taskProvider);
}
})
.isPresent();
}
// Add a task to create merged runtime classes if this is a dynamic-feature,
// or a base module consuming feature jars. Merged runtime classes are needed if code
// minification is enabled in a project with features or dynamic-features.
if (componentProperties.getVariantType().isDynamicFeature()
|| variantScope.consumesFeatureJars()) {
taskFactory.register(new MergeClassesTask.CreationAction(componentProperties));
}
// ----- Minify next -----
/ / confusion
// ----- multi-dex supports...
/ / create the dex
createDexTasks(
creationConfig, componentProperties, dexingType, registeredExternalTransform);
/ /... Resource compression, etc.
}
Copy the codeBefore the Transform, it also performs some Java8 desugmenting and merging Tasks of Java resources, which are added to the original data stream.
Step 5 Create Task for Transfrom
First, we need to understand that there are two types of Transform:
- Consumable: Needs to output the data source to the next Transform
- Reference type: only need to read, no output
This brings us to the logic we care about handling the Transform.
The system will find all the transforms registered in BaseExtension and walk through them, using the transformManager addTransform.
public <T extends Transform> Optional<TaskProvider<TransformTask>> addTransform(
@NonNull TaskFactory taskFactory,
@NonNull ComponentPropertiesImpl componentProperties,
@NonNull T transform,
@Nullable PreConfigAction preConfigAction,
@Nullable TaskConfigAction<TransformTask> configAction,
@Nullable TaskProviderCallback<TransformTask> providerCallback) {
/ /... omit
List<TransformStream> inputStreams = Lists.newArrayList();
String taskName = componentProperties.computeTaskName(getTaskNamePrefix(transform));
// get referenced-only streams
List<TransformStream> referencedStreams = grabReferencedStreams(transform);
// find input streams, and compute output streams for the transform.
IntermediateStream outputStream =
findTransformStreams(
transform,
componentProperties,
inputStreams,
taskName,
componentProperties.getGlobalScope().getBuildDir());
/ /... Testing work
transforms.add(transform);
TaskConfigAction<TransformTask> wrappedConfigAction =
t -> {
t.getEnableGradleWorkers()
.set(
componentProperties
.getGlobalScope()
.getProjectOptions()
.get(BooleanOption.ENABLE_GRADLE_WORKERS));
if(configAction ! =null) { configAction.configure(t); }};// create the task...
return Optional.of(
taskFactory.register(
new TransformTask.CreationAction<>(
componentProperties.getName(),
taskName,
transform,
inputStreams,
referencedStreams,
outputStream,
recorder),
preConfigAction,
wrappedConfigAction,
providerCallback));
}
Copy the codeTaskName = taskName = taskName = taskName
transform${inputType}With${transformName}For${BuildType}
Copy the codeThe taskName rule will be used later.
ReferencedStreams is used to handle referential transforms, so we focus on outputStreams, which are generated by the findTransformStreams method, The problem of data flow is particularly clear in this method:
private final List<TransformStream> streams = Lists.newArrayList();
private IntermediateStream findTransformStreams(
@NonNull Transform transform,
@NonNull ComponentPropertiesImpl componentProperties,
@NonNull List<TransformStream> inputStreams,
@NonNull String taskName,
@NonNull File buildDir) {
/ /...
// Consume data streams. InputStreams adds data streams to consume
// 1. InputStreams consumes streams that streams can consume
consumeStreams(requestedScopes, requestedTypes, inputStreams);
Set<ContentType> outputTypes = transform.getOutputTypes();
File outRootFolder =
FileUtils.join(
buildDir,
StringHelper.toStrings(
AndroidProject.FD_INTERMEDIATES,
FD_TRANSFORMS,
transform.getName(),
componentProperties.getVariantDslInfo().getDirectorySegments()));
// Create an output stream
IntermediateStream outputStream =
IntermediateStream.builder(
project,
transform.getName() + "-" + componentProperties.getName(),
taskName)
.addContentTypes(outputTypes)
.addScopes(requestedScopes)
.setRootLocation(outRootFolder)
.build();
// 2. Add the generated data stream for the next Transform
streams.add(outputStream);
return outputStream;
}
Copy the codeThe process is shown as follows:
This means that each Transform goes through the process in the diagram. For most transforms, the input source of each Transform is the output source of the previous Transform.
So for the developer, if we define a Transform and don’t add the generated file to the output directory, this will result in subsequent transforms not finding the input source, and the compiler will have to report an error.
This is a mistake I made recently.
Going back to the beginning of this step, taskFactory finally registers a TransformTask for us.
Step 6 What does the TransformTask do
Enter the TransformTask class, and there is a method transform that adds the @TaskAction annotation, so this method is called once the Task executes.
@TaskAction
void transform(final IncrementalTaskInputs incrementalTaskInputs)
throws IOException, TransformException, InterruptedException {
// Set incremental compilation
isIncremental.setValue(transform.isIncremental() && incrementalTaskInputs.isIncremental());
// ...
recorder.record(
ExecutionType.TASK_TRANSFORM_PREPARATION,
preExecutionInfo,
getProjectPath().get(),
getVariantName(),
new Recorder.Block<Void>() {
@Override
public Void call(a) throws Exception {
/ /... File processing for incremental compilation
return null; }}); GradleTransformExecution executionInfo = preExecutionInfo.toBuilder().setIsIncremental(isIncremental.getValue()).build(); recorder.record( ExecutionType.TASK_TRANSFORM, executionInfo, getProjectPath().get(), getVariantName(),new Recorder.Block<Void>() {
@Override
public Void call(a) throws Exception {
// ...
transform.transform(
newTransformInvocationBuilder(context) .addInputs(consumedInputs.getValue()) .addReferencedInputs(referencedInputs.getValue()) .addSecondaryInputs(changedSecondaryInputs.getValue()) .addOutputProvider( outputStream ! =null
? outputStream.asOutput()
: null)
.setIncrementalMode(isIncremental.getValue())
.build());
if(outputStream ! =null) {
outputStream.save();
}
return null; }}); }Copy the codeYou don’t care about the Recorder, it’s just an executor that will eventually execute the code in the Block.
If it is an incrementally compiled Task, it will process the files and tell us which files have changed.
After that, the Transform method of the Transform is executed, and the whole Transform ends.
Step 7 DexBuild
Back to step 4, AGP will register the tasks supported by confusion and multiple Dex, and then come to the Task that created the Dex:
private void createDexTasks(
@NonNull ApkCreationConfig apkCreationConfig,
@NonNull ComponentPropertiesImpl componentProperties,
@NonNull DexingType dexingType,
boolean registeredExternalTransform) {
// ...
taskFactory.register(
new DexArchiveBuilderTask.CreationAction(
dexOptions,
enableDexingArtifactTransform,
componentProperties));
/ /...
}
Copy the codeA DexArchiveBuilderTask is a task named dexBuilder, annotated with:
Task that converts CLASS files to dex archives
It is the Task that creates the dex file.
To learn more about Dex, read:
Android Dex File
At this point, our source code analysis is complete.
Third, solve the problem
Before I have always said AGP 3. 7.0.x.x can hook to transformClassesWithDexBuilderForXXX task, the AGP 4. 7.0.x.x wouldn’t work.
Carefully look at me the taskName naming rules mentioned above, you will find, in 3. Before 7.0.x.x transformClassesWithDexBuilderForXXX is a Transform, I remember the corresponding class DexTransform, It will help AGP generate.dex files.
In 4.1.1, this task is assigned to the DexArchiveBuilderTask, which is no longer a Transform.
So, it’s not uncommon to see Android developers say, “Oh, my God, AGP version has been updated, my method doesn’t work!”
Therefore, it is concluded that on AGP, it is better not to hook source code, and it is recommended to use the interface recommended by the official.
conclusion
The content of this article is actually a verification of the above Transform process, I believe you have a general grasp of the Transform process!
If there is any controversial content, welcome to leave a message in the comment section, if you think this article is good, “like” is the biggest affirmation of this article!
Article quote:
Bytecode in Android Projects