What are AAPT2

During Android development, we start a build task using the Gradle command, which eventually generates the build product “APK” file. The normal APK construction process is as follows:

(Quoted from official Google docs)

Compile all resource files, generate resource tables and R files;
Compile Java files and package class files as dex files;
Package resources and dex files to generate unsigned APK files;
Signature APK generates a formal package.

Older versions of Android use the AAPT compiler for resource compilation by default. Since Android Studio 3.0, AS has enabled AAPT2 AS the compiler for resource compilation by default. Currently, AAPT2 is also the mainstream trend of Android development. Learning the working principle of AAPT2 can help Android developers better grasp the APK construction process, so as to help solve the problems encountered in actual development.

The AAPT2 executable files are shipped with the Build Tools of the Android SDK. The AAPT2 Tools are included in the Build-Tools folder of Android Studio. The directory is SDK directory /build-tools/version/ AAPT2.

How do AAPT2 work

While looking at the Android build process, I can’t help but wonder:

I understand that Java files need to be compiled to generate class files, but what exactly does resource file compilation do? Why compile resources?

With that in mind, let’s take a closer look at AAPT2. Unlike AAPT, AAPT2 splits the resource compilation and packaging process into two parts, compilation and linking:

Compilation: Compiles a resource file to a binary file (flat).

Link: Merge compiled files and package them into separate files.

By splitting resource compilation into two parts, AAPT2 can improve the performance of resource compilation. For example, if a resource file has changed before, AAPT needs to do a full compilation. AAPT2 only need to recompile the changed file and then link with other files that have not changed.

2.1 Compile command

As described above, the Complie directive is used to Compile resources, and AAPT2 provide several options to use with the Compile command.

The common uses of Complie are as follows:

aapt2 compile path-to-input-files [options] -o output-directory/
Copy the code

After executing the command, AAPT2 will compile the resource file into a. Flat format file. The file comparison is as follows.

The Compile command verifies the path of the resource file. The path of the input file must conform to the following structure: path/resource-type[-config]/file.

For example, if resource files are saved in the aapT2 folder and are compiled using the Compile command, error: invalid file path ‘… / aapt2 / ic_launcher. PNG ‘”. Change aAPT to “drawable-hdpi”, compile ok.

In Android in the Studio, can be in app/build/intermediates/res/merged/directory to find a compiler generated. Flat file. Of course, Compile also supports compiling multiple files;

aapt2 compile path-to-input-files1 path-to-input-files2 [options] -o output-directory/
Copy the code

To compile the entire directory, you need to make data files. The compiled product is a compressed file, containing all resources in the directory. The resource directory structure is flattened by the file name.

aapt2 compile --dir ... /res [options] -o output-directory/resource.ap_Copy the code

You can see the compiled resource file (PNG, XML…) The FLAT file will be compiled into a FLAT file. Drag the FLAT file into the AS to open it. It is garbled. So what exactly is this FLAT file?

2.2 FLAT file

A FLAT file is a generated file compiled by AAPT2, also known as AAPT2 container. A file consists of a header and a resource item:

The file header

Resource item

Resource entries are divided into two types based on entry_type values:

When entry_type is equal to 0x00000000, the entry_type is RES_TABLE.
When entry_type is equal to 0x00000001, the value is RES_FILE.

RES_TABLE contains the ResourceTable structure in protobuf format. The data structure is as follows:

// Top level message representing a resource table.
message ResourceTable {
    // String pool
    StringPool source_pool = 1;
    // Used to generate resource ids
    repeated Package package = 2;
    // Resource layer correlation
    repeated Overlayable overlayable = 3;
    // Tool version
    repeated ToolFingerprint tool_fingerprint = 4;
}
Copy the code

The ResourceTable table contains:

StringPool: string pool. String constant pool is used to reuse strings in resource files to reduce volume. The corresponding strings in resource files are replaced with indexes in the string pool.

message StringPool {
  bytes data = 1;
}
Copy the code

Package: contains information about resource ids

// The package ID part of the resource ID is in the range [0x00, 0xff]
message PackageId {
  uint32 id = 1;
}
// Naming rules for resource ids
message Package {
  // [0x02, 0x7f) simply put, used by the system
  // 0x7f Application use
  // (0x7f, 0xff] Reserved Id
  PackageId package_id = 1;
  / / package name
  string package_name = 2;
  // Resource type, corresponding to String, layout, XML, dimen, attr, etc., whose corresponding resource ID range is 0x01, 0xFF.
  repeated Type type = 3;
}
Copy the code

The command for resource ID follows the 0xPPTTEEEE rule, where PP corresponds to the PackageId, the resource used by common applications is 7F, TT corresponds to the name of the resource folder, and the last four digits are the resource ID, starting from 0.

The format of RES_FILE is as follows:

The RES_FILE FLAT file structure can be seen in the following figure.

As you can see from the file formats shown in the figure above, a FLAT can contain multiple resource items. In resource items, the Header field holds the contents of the CompiledFile serialized in the Protobuf format. In this structure, information such as file name, file path, file configuration, and file type is saved. The data field holds the contents of the resource file. In this way, a file contains both external information about the file and the original content of the file.

2.3 Compiled source code

Above, we learned the usage of Compile command and the file format of the compilation product FLAT file. Next, we learn the compilation process of AAPT2 from the source level by looking at the code. The source address of this article is the source address.

2.3.1 Command Execution Process

If you open main. CPP, you can find the main entry.

int main(int argc, char** argv) {
#ifdef _WIN32
    ......
    // Parameter format conversion
    argv = utf8_argv.get();
#endif
    // Specific implementation in MainImpl
    return MainImpl(argc, argv);
}
Copy the code

In MainImpl, you first get the parameter part from the input, and then create a MainCommand to execute the command.

int MainImpl(int argc, char** argv) {
    if (argc < 1) {
        return -1;
    }
    // Input from subscript 1 is stored in args
    std::vector<StringPiece> args;
    for (int i = 1; i < argc; i++) {
        args.push_back(argv[i]);
    }
    // Omit some code for printing information and error handling
    // Create a MainCommand
    aapt::MainCommand main_command(&printer, &diagnostics);
    // AapT2 daemon mode,
    main_command.AddOptionalSubcommand( aapt::util::make_unique<aapt::DaemonCommand>(&fout, &diagnostics));
    // Call the Execute method to Execute the command
    return main_command.Execute(args, &std::cerr);
}
Copy the code

MainCommand inherits from Command. Multiple secondary commands are added to the initialization method of MainCommand. Based on the class name, you can easily infer that these commands correspond to the secondary commands viewed by the terminal through the Command.

explicit MainCommand(text::Printer* printer, IDiagnostics* diagnostics)
 : Command("aapt2").diagnostics_(diagnostics) {
    // corresponds to the Compile command
    AddOptionalSubcommand(util::make_unique<CompileCommand>(diagnostics));
    // Corresponds to the link command
    AddOptionalSubcommand(util::make_unique<LinkCommand>(diagnostics));
    AddOptionalSubcommand(util::make_unique<DumpCommand>(printer, diagnostics));
    AddOptionalSubcommand(util::make_unique<DiffCommand>());
    AddOptionalSubcommand(util::make_unique<OptimizeCommand>());
    AddOptionalSubcommand(util::make_unique<ConvertCommand>());
    AddOptionalSubcommand(util::make_unique<VersionCommand>());
}
Copy the code

The AddOptionalSubcommand method is defined in the base Command class. The content is relatively simple. The subCommand passed in is stored in an array.

void Command::AddOptionalSubcommand(std::unique_ptr<Command>&& subcommand, bool experimental) {
    subcommand->full_subcommand_name_ = StringPrintf("%s %s", name_.data(), subcommand->name_.data());
    if (experimental) {
        experimental_subcommands_.push_back(std::move(subcommand));
    } else{ subcommands_.push_back(std::move(subcommand)); }}Copy the code

Main_command.Execute () main_command.Execute () main_command. There is no Execute method implemented in MainCommand, it should be implemented in the parent class, then search in the Command class, sure enough, there.

int Command::Execute(const std::vector<StringPiece>& args, std::ostream* out_error) {
    TRACE_NAME_ARGS("Command::Execute", args);
    std::vector<std::string> file_args;
    for (size_t i = 0; i < args.size(); i++) {
        const StringPiece& arg = args[i];
        // The argument is not '-'
        if(*(arg.data()) ! =The '-') {
        // is the first argument
        if (i == 0) {
            for (auto& subcommand : subcommands_) {
                // Check if it is a subcommand
                if(arg == subcommand->name_ || (! subcommand->short_name_.empty() && arg == subcommand->short_name_)) {// Executes the Execute method of the subcommand, moving the passed argument back one bit
                return subcommand->Execute( std::vector<StringPiece>(args.begin() + 1, args.end()), out_error); }}// Omit some code
    // Call the Action method. When executing the secondary command, file_args holds the displaced argument
    return Action(file_args);
}
Copy the code

In the Execute method, parameters are evaluated first, if the first parameter matches the second level command (Compile, link,…..). , the Execute method of the second-level command is invoked. As shown in the preceding command compilation example, the Execute method of the second-level command is invoked after the judgment of the second-level command is matched.

In the command-cpp directory, you can find Compile. CPP, whose Execute inherits from the parent class. But because the argument has already been shifted, the Action method is eventually executed. The Action method can be found in Compile. CPP, as well as in the implementation classes of other secondary commands (link. CPP, dump.cpp…). , the core processing is also in the Action method. The overall call is shown as follows:

Before we look at the Action code, let’s look at the CompileCommand header. When CompileCommand is initialized, it defines both mandatory and optional parameters.

SetDescription("Compiles resources to be linked into an apk.");
AddRequiredFlag("-o"."Output path", &options_.output_path, Command::kPath);
AddOptionalFlag("--dir"."Directory to scan for resources", &options_.res_dir, Command::kPath);
AddOptionalFlag("--zip"."Zip file containing the res directory to scan for resources", &options_.res_zip, Command::kPath);
AddOptionalFlag("--output-text-symbols"."Generates a text file containing the resource symbols in the\n" "specified file", &options_.generate_text_symbols_path, Command::kPath);
AddOptionalSwitch("--pseudo-localize"."Generate resources for pseudo-locales " "(en-XA and ar-XB)", &options_.pseudolocalize);
AddOptionalSwitch("--no-crunch"."Disables PNG processing", &options_.no_png_crunch);
AddOptionalSwitch("--legacy"."Treat errors that used to be valid in AAPT as warnings", &options_.legacy_mode);
AddOptionalSwitch("--preserve-visibility-of-styleables"."If specified, apply the same visibility rules for\n" "styleables as are used for all other resources.\n" "Otherwise, all stylesables will be made public.", &options_.preserve_visibility_of_styleables);
AddOptionalFlag("--visibility"."Sets the visibility of the compiled resources to the specified\n" "level. Accepted levels: public, private, default", &visibility_);
AddOptionalSwitch("-v"."Enables verbose logging", &options_.verbose);
AddOptionalFlag("--trace-folder"."Generate systrace json trace fragment to specified folder.", &trace_folder_);
Copy the code

The compile options listed on the official website are not complete. If you print the information using compile-h, you will find that the printed information is consistent with the Settings in the code.

The execution flow of the Action method can be summarized as follows:

1) The system determines the resource type according to the input parameter and creates the corresponding file loader (File_collection).

2) Determine the type of the output file according to the incoming output path, and create the corresponding archive_writer. Archive_writer is passed down in the subsequent call chain, and the compiled file is finally written to the output directory through archive_writer.

3) Call Compile method to execute compilation.

The file read and write objects involved in procedure 1,2 are listed below.

The simplified main process code is as follows:

int CompileCommand::Action(const std::vector<std::string>& args) {
    // omit some code....
    std::unique_ptr<io::IFileCollection> file_collection;
    // Load the input resource, simplify the logic, and omit the verification code below
    if (options_.res_dir && options_.res_zip) {
        context.GetDiagnostics()->Error(DiagMessage() << "only one of --dir and --zip can be specified");
        return 1;
    } else if (options_.res_dir) {
        // Load the resource file in the directory...
        file_collection = io::FileCollection::Create(options_.res_dir.value(), &err);
        / /...
    }else if (options_.res_zip) {
        // Load resource files in compressed package format...
        file_collection = io::ZipFileCollection::Create(options_.res_zip.value(), &err);
        / /...
    } else {
        FileCollection = FileCollection = FileCollection = FileCollection = FileCollection
        file_collection = std::move(collection);
    }
    std::unique_ptr<IArchiveWriter> archive_writer;
    // Product output file type
    file::FileType output_file_type = file::GetFileType(options_.output_path);
    if (output_file_type == file::FileType::kDirectory) {
        // Output to the file directory
        archive_writer = CreateDirectoryArchiveWriter(context.GetDiagnostics(), options_.output_path);
    } else {
        // Output to compressed package
        archive_writer = CreateZipFileArchiveWriter(context.GetDiagnostics(), options_.output_path);
    }
    if(! archive_writer) {return 1;
    }
    return Compile(&context, file_collection.get(), archive_writer.get(), options_);
}
Copy the code

The Compile method compiles the input resource file names. Each resource file is processed as follows:

Parses the input resource path to obtain information such as resource name and extension.
Determine the file type based on path and set different compile_func.
Generate the file name of the output. The output is the FLAT file name, which will be spliced across the full path to generate a file name similar to the one in the previous example — “drawable- hdpi_IC_launcher.png. FLAT”.
Pass in the parameters and call compile_func.

ResourcePathData contains information such as resource paths, resource names, and resource extensions. AAPT2 obtain resource types from ResourcePathData.

 int Compile(IAaptContext* context, io::IFileCollection* inputs, IArchiveWriter* output_writer, CompileOptions& options) {
    TRACE_CALL();
    bool error = false;
    // Compile the input resource file
    auto file_iterator  = inputs->Iterator();
    while (file_iterator->HasNext()) {
        // omit some code (file verification related...)
        std::string err_str;
        ResourcePathData path_data;
        // Get the full path name for subsequent file type determination
        if (auto maybe_path_data = ExtractResourcePathData(path, inputs->GetDirSeparator(), &err_str)) {
            path_data = maybe_path_data.value();
        } else {
            context->GetDiagnostics()->Error(DiagMessage(file->GetSource()) << err_str);
            error = true;
            continue;
        }
 
        // Depending on the file type, select the method to CompileFile, where the function pointer points to a compilation method.
        // use use the method set to CompileFile
        auto compile_func = &CompileFile;
        // If it is an XML resource from values, use CompileTable and change the extension to arsc
        if (path_data.resource_dir == "values" && path_data.extension == "xml") {
            compile_func = &CompileTable;
            // We use a different extension (not necessary anymore, but avoids altering the existing // build system logic).
            path_data.extension = "arsc";
        } else if (const ResourceType* type = ParseResourceType(path_data.resource_dir)) {
            // Parse the resource type, if the type is kRaw, execute the default compilation method, otherwise execute the following code.
            if(*type ! = ResourceType::kRaw) {// XML path or file extension to.xml
                if (*type == ResourceType::kXml || path_data.extension == "xml") {
                    // XML class, CompileXml method
                    compile_func = &CompileXml;
                } else if((! options.no_png_crunch && path_data.extension =="png") || path_data.extension == "9.png") { // If the suffix is.png and turn on PNG optimization or a dot 9 image type
                    // PNG class, CompilePng methodcompile_func = &CompilePng; }}}else {
            // Invalid type, output error message, continue loop
            context->GetDiagnostics()->Error(DiagMessage() << "invalid file path '" << path_data.source << "'");
            error = true;
            continue;
        } 
        // Check whether the file name exists.
        if(compile_func ! = &CompileFile && ! options.legacy_mode && std::count(path_data.name.begin(), path_data.name.end(),'. ') != 0) {
            error = true;
            context->GetDiagnostics()->Error(DiagMessage(file->GetSource()) << "file name cannot contain '.' other than for" << " specifying the extension");
            continue;
        }
        // Generate the product file name. This method generates the finished flat file name, such as drawable- hdpi_IC_launcher.png
        const std::string out_path = BuildIntermediateContainerFilename(path_data);
        // Execute the compile method
        if(! compile_func(context, options, path_data, file, output_writer, out_path)) { context->GetDiagnostics()->Error(DiagMessage(file->GetSource()) <<"file failed to compile"); error = true; }}return error ? 1 : 0;
}
Copy the code

There are four compiler functions for different resource types:

CompileFile
CompileTable
CompileXml
CompilePng

XML files in the RAW directory will not be CompileXml. Resources in the RAW directory are copied directly to APK without XML optimization compilation. In addition to CompileTable, resources in the VALUES directory will also change the name extension of resource files. You can assume that other compilations will more or less handle raw resources and then compile them in FLAT files. The process of this part is shown in the figure below:

The main flow of compiling command execution ends here. From source code analysis, we can see that AAPT2 compile input files into FLAT files. Below, we take a closer look at four compilation methods.

2.3.2 Four compiler functions

CompileFile

Function to construct ResourceFile object and the original file data, and then call WriteHeaderAndDataToWriter writes the data in the output file (flat).

static bool CompileFile(IAaptContext* context, const CompileOptions& options, const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer, const std::string& output_path) {
    TRACE_CALL();
    if (context->IsVerbose()) {
        context->GetDiagnostics()->Note(DiagMessage(path_data.source) << "compiling file");
    }
    // Define the ResourceFile object and save the config, source and other information
    ResourceFile res_file;
    res_file.name = ResourceName({}, *ParseResourceType(path_data.resource_dir), path_data.name);
    res_file.config = path_data.config;
    res_file.source = path_data.source;
    res_file.type = ResourceFile::Type::kUnknown; // This type may have XML, PNG, or something else, but set the type to unknow.
    // Raw file data
    auto data = file->OpenAsData();
    if(! data) { context->GetDiagnostics()->Error(DiagMessage(path_data.source) <<"failed to open file ");
        return false;
    }
    return WriteHeaderAndDataToWriter(output_path, res_file, data.get(), writer, context->GetDiagnostics());
}
Copy the code

ResourceFile content is relatively simple, finishing the assignment of file information is invoked by WriteHeaderAndDataToWriter method.

WriteHeaderAndDataToWriter in this method, the previously created archive_writer (can be searched in this paper, the file write to create complete, would have been handed down to) make a packing, The packaged ContainerWriter has both regular file write and protobuf serialized write capabilities.

Pb provides ZeroCopyStream interface for user data reading and writing and serialization/deserialization operations.

WriteHeaderAndDataToWriter simple process can be summarized as:

IArchiveWriter StartEntry, open the file, and prepared to write;
ContainerWriter AddResFileEntry, write data;
IArchiveWriter FinishEntry, close the file, free memory.

static bool WriteHeaderAndDataToWriter(const StringPiece& output_path, const ResourceFile& file, io::KnownSizeInputStream* in, IArchiveWriter* writer, IDiagnostics* diag) {
    // Open the file
    if(! writer->StartEntry(output_path,0)) {
        diag->Error(DiagMessage(output_path) << "failed to open file");
        return false;
    }
    // Make sure CopyingOutputStreamAdaptor is deleted before we call writer->FinishEntry().
    {
        // Wrap write to write protobuf data
        CopyingOutputStreamAdaptor copying_adaptor(writer);
        ContainerWriter container_writer(© ing_adaptor, 1 u);
        // Serialize file to the protobuf format. The serialized file is pb_compiled_file. The file file is ResourceFile, which contains the original file path and configuration information
        pb::internal::CompiledFile pb_compiled_file;
        SerializeCompiledFileToPb(file, &pb_compiled_file);
        // Write pb_compiled_file and in (the original file) to the production file
        if(! container_writer.AddResFileEntry(pb_compiled_file, in)) { diag->Error(DiagMessage(output_path) <<"failed to write entry data");
            return false; }}// Exit the write state
    if(! writer->FinishEntry()) { diag->Error(DiagMessage(output_path) <<"failed to finish writing data");
        return false;
    }
    return true;
}
Copy the code

In archive. CPP, the ZipFileWriter and DirectoryWriter implementations are somewhat different, but logically the same. Only the implementation of DirectoryWriter will be analyzed here.

StartEntry, call fopen to open the file.

bool StartEntry(const StringPiece& path, uint32_t flags) override {
    if (file_) {
        return false;
    }
    std::string full_path = dir_;
    file::AppendPath(&full_path, path);
    file::mkdirs(file::GetStem(full_path).to_string());
    // Open the file
    file_ = {::android::base::utf8::fopen(full_path.c_str(), "wb"), fclose};
    if(! file_) { error_ = SystemErrorCodeToString(errno);return false;
    }
    return true;
}
Copy the code

FinishEntry, call reset to free memory.

bool FinishEntry(a) override {
    if(! file_) {return false;
    }
    file_.reset(nullptr);
    return true;
}
Copy the code

The ContainerWriter class is defined in the container. CPP class file. In the constructor of the ContainerWriter class, you can find the code for writing the file header in the same format as described in the “FLAT Format” section above.

// In the constructor of the class, write the information of the file header
ContainerWriter::ContainerWriter(ZeroCopyOutputStream* out, size_t entry_count)
  : out_(out), total_entry_count_(entry_count), current_entry_count_(0u) {
    CodedOutputStream coded_out(out_);
    // Magic data, kContainerFormatMagic = 0x54504141U
    coded_out.WriteLittleEndian32(kContainerFormatMagic);  
    KContainerFormatVersion = 1u
    coded_out.WriteLittleEndian32(kContainerFormatVersion);
    Total_entry_count_ is assigned when ContainerReader is constructed, and the value is passed in externally
    coded_out.WriteLittleEndian32(static_cast<uint32_t>(total_entry_count_));
    if (coded_out.HadError()) {
        error_ = "failed writing container format header"; }}Copy the code

Call ContainerWriter’s AddResFileEntry method and write the content of the resource entry.

// file: indicates information files in protobuf format. In: indicates original files
bool ContainerWriter::AddResFileEntry(const pb::internal::CompiledFile& file, io::KnownSizeInputStream* in) {
    // Check the number of items. If the number is greater than the set number, an error is reported directly
    if (current_entry_count_ >= total_entry_count_) {
        error_ = "too many entries being serialized";
        return false;
    }
    / / entries + +
    current_entry_count_++;
    constexpr const static int kResFileEntryHeaderSize = 12; ,/ / the output stream
    CodedOutputStream coded_out(out_);
    // Write the resource type
    coded_out.WriteLittleEndian32(kResFile);
 
    const ::google::protobuf::uint32
    // ResourceFile File length. This part contains information such as the path, type, and configuration of the current file
    header_size = file.ByteSize();
    const int header_padding = CalculatePaddingForAlignment(header_size);
    // The length of the original file
    const ::google::protobuf::uint64 data_size = in->TotalSize();
    const int data_padding = CalculatePaddingForAlignment(data_size);
    KResFileEntryHeaderSize (fixed 12) + ResourceFile file length + header_PADDING + original file length + data_padding
    coded_out.WriteLittleEndian64(kResFileEntryHeaderSize + header_size + header_padding + data_size + data_padding);
 
    // Write the length of the header
    coded_out.WriteLittleEndian32(header_size);
    // Write data length
    coded_out.WriteLittleEndian64(data_size);
    // Write "header"
    file.SerializeToCodedStream(&coded_out);
    / / alignment
    WritePadding(header_padding, &coded_out);
    Before using Copy, you need to call Trim (why is not clear, but let's learn AAPT2 and understand the underlying API function). If any readers know, please comment)
    coded_out.Trim();
    // exception judgment
    if (coded_out.HadError()) {
        error_ = "failed writing to output"; return false;
    } if(! io::Copy(out_, in)) {// Resource data payload (PNG, XML, XmlNode)
        if (in->HadError()) {
            std::ostringstream error;
            error << "failed reading from input: " << in->GetError();
            error_ = error.str();
        } else {
            error_ = "failed writing to output";
        }
        return false;
    }
    / / for its
    WritePadding(data_padding, &coded_out);
    if (coded_out.HadError()) {
        error_ = "failed writing to output";
        return false;
    }
    return true;
}
Copy the code

The FLAT file is then written, and the output file contains not only the resource content, but also the file name, path, and configuration.

CompilePng

This method is similar to CompileFile except that PNG images are processed first (PNG optimization and image 9 processing) and then FLAT files are written to them.

static bool CompilePng(IAaptContext* context, const CompileOptions& options, const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer, const std::string& output_path) {
    / /.. Omit some verification code
    BigBuffer buffer(4096);
   // Same type of code with different type
    ResourceFile res_file;
    res_file.name = ResourceName({}, *ParseResourceType(path_data.resource_dir), path_data.name);
    res_file.config = path_data.config;
    res_file.source = path_data.source;
    res_file.type = ResourceFile::Type::kPng;
 
    {
        // Read the resource contents into data
        auto data = file->OpenAsData();
        // Read the result to verify
        if(! data) { context->GetDiagnostics()->Error(DiagMessage(path_data.source) <<"failed to open file ");
            return false;
        }
        // Save the output stream
        BigBuffer crunched_png_buffer(4096);
        io::BigBufferOutputStream crunched_png_buffer_out(&crunched_png_buffer);
 
        // Optimize the PNG image
        const StringPiece content(reinterpret_cast<const char*>(data->data()), data->size(a));
        PngChunkFilter png_chunk_filter(content);
        std::unique_ptr<Image> image = ReadPng(context, path_data.source, &png_chunk_filter);
        if(! image) {return false;
        }
         
        // process
        std::unique_ptr<NinePatch> nine_patch;
        if (path_data.extension == "9.png") {
            std::string err;
            nine_patch = NinePatch::Create(image->rows.get(), image->width, image->height, &err);
            if(! nine_patch) { context->GetDiagnostics()->Error(DiagMessage() << err);return false;
            }
            // Remove the 1 pixel border
            image->width -= 2;
            image->height -= 2;
            memmove(image->rows.get(), image->rows.get() + 1, image->height * sizeof(uint8_t**));
            for (int32_t h = 0; h < image->height; h++) {
                memmove(image->rows[h], image->rows[h] + 4, image->width * 4);
            } if (context->IsVerbose()) {
                context->GetDiagnostics()->Note(DiagMessage(path_data.source) << "9-patch: "<< *nine_patch); }}// Save the processed PNG to &crunched_png_buffer_out
        if(! WritePng(context, image.get(), nine_patch.get(), &crunched_png_buffer_out, {})) {return false;
        }
 
        / /... Omit part of the image verification code, this part of the code will compare the size of the optimized image and the original image, if the optimized image is larger than the original image, the original image will be used. (PNG optimization may be larger than the original image)
    }
    io::BigBufferInputStream buffer_in(&buffer);
    // Call the same method to CompileFile and write to the flat file whose content is
    return WriteHeaderAndDataToWriter(output_path, res_file, &buffer_in, writer, context->GetDiagnostics());
}
Copy the code

AAPT2’s compression of PNG images can be divided into three aspects:

Whether RGB can be converted into grayscale;
Whether the transparent channel can be deleted;
Is it 256 colors at most (Indexed_color optimization)?

PNG optimization, interested students can have a look

In PNG processing finished, also called WriteHeaderAndDataToWriter to write the data, this part can be reading the above analysis, no longer here.

CompileXml

This method parses the XML and then creates an XmlResource, which contains the resource name, configuration, type, and so on. Write an output file by a flattener XMLTooutstream function.

static bool CompileXml(IAaptContext* context, const CompileOptions& options,
                       const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer,
                       const std::string& output_path) {
  / /... Omit check code
  std::unique_ptr<xml::XmlResource> xmlres;
  {
    // Open the XML file
    auto fin = file->OpenInputStream();
    / /... Omit check code
    / / to parse the XML
    xmlres = xml::Inflate(fin.get(), context->GetDiagnostics(), path_data.source);
    if(! xmlres) {return false; }}//
  xmlres->file.name = ResourceName({}, *ParseResourceType(path_data.resource_dir), path_data.name);
  xmlres->file.config = path_data.config;
  xmlres->file.source = path_data.source;
  xmlres->file.type = ResourceFile::Type::kProtoXml;
 
  // Check whether the resource with the ID type has a valid ID (if there is an abnormal ID, if "has an invalid entry name" is displayed)
  XmlIdCollector collector;
  if(! collector.Consume(context, xmlres.get())) {return false;
  }
 
  // Handle aapt:attr embedded resources
  InlineXmlFormatParser inline_xml_format_parser;
  if(! inline_xml_format_parser.Consume(context, xmlres.get())) {return false;
  }
 
  // Open the output file
  if(! writer->StartEntry(output_path,0)) {
    context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to open file");
    return false;
  }
 
  std::vector<std::unique_ptr<xml::XmlResource>>& inline_documents =
      inline_xml_format_parser.GetExtractedInlineXmlDocuments();
 
  {
    // Similar to CompileFile, create a writer that can handle Protobuf serialization
    CopyingOutputStreamAdaptor copying_adaptor(writer);
    ContainerWriter container_writer(© ing_adaptor, 1 u + inline_documents. The size ());
 
    if(! FlattenXmlToOutStream(output_path, *xmlres, &container_writer, context->GetDiagnostics())) {return false;
    }
    // Handle embedded elements (aapt:attr)
    for (const std::unique_ptr<xml::XmlResource>& inline_xml_doc : inline_documents) {
      if(! FlattenXmlToOutStream(output_path, *inline_xml_doc, &container_writer, context->GetDiagnostics())) {return false; }}}// Free memory
  if(! writer->FinishEntry()) { context->GetDiagnostics()->Error(DiagMessage(output_path) <<"failed to finish writing data");
    return false;
  }
  // The compile options section is omitted
  return true;
}
Copy the code

In the compile XML method, instead of creating a ResourceFile as in the previous two methods, an XmlResource is created to hold information about the XML resource. Its structure contains the following:

After the Inflate method is executed, the XmlResource contains information about the resource and the DOM tree of the XML. InlineXmlFormatParser is used to parse the inline attribute aapt:attr.

Using AAPT’s embedded resource format, you can define all multiple resources in the same XML file, which is more compact if resource reuse is not required. The XML tag tells AAPT that the tag’s children should be treated as resources and extracted into their own resource files. The value in the attribute name is used to specify where to use the embedded resource within the parent tag. AAPT generates resource files and names for all embedded resources. Applications built using this embedded format are compatible with all versions of Android. — Official documents

The resolved FlattenXmlToOutStream will call SerializeCompiledFileToPb method, first the resource file information into protobuf format, We then call SerializeXmlToPb to convert the previously parsed Element node information into AN XmlNode (protobuf structure, also defined in Resources), and then convert the generated XmlNode into a string. Finally, it is added to the resource item of the FLAT file through the AddResFileEntry method above. It can be seen that a FLAT file generated through XML can contain multiple resource items in a single FLAT file.

static bool FlattenXmlToOutStream(const StringPiece& output_path, const xml::XmlResource& xmlres,
                                  ContainerWriter* container_writer, IDiagnostics* diag) {
  // Serialize the CompiledFile section
  pb::internal::CompiledFile pb_compiled_file;
  SerializeCompiledFileToPb(xmlres.file, &pb_compiled_file);
 
  // Serialize the XmlNode part
  pb::XmlNode pb_xml_node;
  SerializeXmlToPb(*xmlres.root, &pb_xml_node);
 
  // String format stream, here can look at the source code
  std::string serialized_xml = pb_xml_node.SerializeAsString();
  io::StringInputStream serialized_in(serialized_xml);
  // Save to the resource entry
  if(! container_writer->AddResFileEntry(pb_compiled_file, &serialized_in)) { diag->Error(DiagMessage(output_path) <<"failed to write entry data");
    return false;
  }
  return true;
}
Copy the code

Protoserializexmltopb in Protoserialize.cpp, which implements node structure replication through traversal and recursion, interested readers can view the source code.

CompileTable

The CompileTable function handles resources under Values. As you can see, resources under Values are compilable to arSC. The final output file is named *.arsc.flat, and the result is as follows:

At the beginning of the function, the resource file is read, the XML is parsed and saved as ResourceTable, and then converted to pb::ResourceTable in protobuf format via SerializeTableToPb, SerializeWithCachedSizes is then called to serialize the Protobuf table to the output file.

static bool CompileTable(IAaptContext* context, const CompileOptions& options,
                         const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer,
                         const std::string& output_path) {
  // Filenames starting with "donottranslate" are not localizable
  bool translatable_file = path_data.name.find("donottranslate") != 0;
  ResourceTable table;
  {
    // Read the file
    auto fin = file->OpenInputStream();
    if (fin->HadError()) {
      context->GetDiagnostics()->Error(DiagMessage(path_data.source)
          << "failed to open file: " << fin->GetError());
      return false;
    }
 
    // Create an XmlPullParser and set up many handlers for XML parsing
    xml::XmlPullParser xml_parser(fin.get());
     
    // Set parsing optionsResourceParserOptions parser_options; parser_options.error_on_positional_arguments = ! options.legacy_mode; parser_options.preserve_visibility_of_styleables = options.preserve_visibility_of_styleables; parser_options.translatable = translatable_file; parser_options.visibility = options.visibility;// Create ResourceParser and save the result to ResourceTable
    ResourceParser res_parser(context->GetDiagnostics(), &table, path_data.source, path_data.config,
        parser_options);
    // Perform parsing
    if(! res_parser.Parse(&xml_parser)) {return false; }}// Omit some verification code
 
  // Open the output file
  if(! writer->StartEntry(output_path,0)) {
    context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to open");
    return false;
  }
 
  {
    // As before, create ContainerWriter for writing files
    CopyingOutputStreamAdaptor copying_adaptor(writer);
    ContainerWriter container_writer(© ing_adaptor, 1 u);
 
    pb::ResourceTable pb_table;
    // Serialize ResourceTable to pb::ResourceTable
    SerializeTableToPb(table, &pb_table, context->GetDiagnostics());
    // Write data entry pb::ResourceTable
    if(! container_writer.AddResTableEntry(pb_table)) { context->GetDiagnostics()->Error(DiagMessage(output_path) <<"failed to write");
      return false; }}if(! writer->FinishEntry()) { context->GetDiagnostics()->Error(DiagMessage(output_path) <<"failed to finish entry");
    return false;
  }
  / /... Omit some code...
  }
 
  return true;
}
Copy the code

Questions and conclusions

AAPT2 is an Android resource packaging build tool that divides resource compilation into compilation and linking. Compilation is the unified compilation of different resource files to generate a binary format optimized for the Android platform. The FLAT file contains not only the content of the original resource file, but also information about the source and type of the resource. Such a file contains all the information required by the resource and is coupled with other dependencies.

At the beginning of this article, we had the following question:

Java files need to be compiled to create.class files, which I can understand, but what exactly does resource file compilation do? Why compile resources?

Therefore, the answer of this paper is: AAPT2 compile resource files into FLAT files, and it can be seen from the file structure of resource items that part of the data in the FLAT file is the original resource content and part is related information of the file. When compiled, the resulting intermediate file contains comprehensive information that can be used for incremental compilation. In addition, some sources on the web suggest that binary resources are smaller and load faster.

AAPT2 compile resource files into FLAT files through compilation, and then link to generate R files and resource tables. Due to space problems, the linking process will be analyzed in the next article.

Iv. Reference documents

1.juejin.cn

2.github.com

3.booster.johnsonlee.io

Author: Shi Xiang, Vivo Internet Front End Team

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Android build tools -AAPT2 source code parsing (1)

What are AAPT2

How do AAPT2 work

2.1 Compile command

2.2 FLAT file

2.3 Compiled source code

2.3.1 Command Execution Process

2.3.2 Four compiler functions

Questions and conclusions

Iv. Reference documents

Android build tools -AAPT2 source code parsing (1)

What are AAPT2

How do AAPT2 work

2.1 Compile command

2.2 FLAT file

2.3 Compiled source code

2.3.1 Command Execution Process

2.3.2 Four compiler functions

Questions and conclusions

Iv. Reference documents

Related Posts

Play MMKV storage tool (Android)

RxJava Meditation (iii) : Time dimension

Understanding the Java Memory Model for the Java Virtual Machine (2)