preface
This section we will revolve around the alipay App build optimizing parsing the qixin series, segmentation and dismantling the client in the “code management”, “certificate management”, “version management”, the construction of the “package” discusses the dimensions of the concrete implementation plan, lead us to learn more about pay treasure in the App under the building blocks of continuous optimization.
This section mainly records the improvement of operation efficiency and quality through the compression of alipay Android package size.
background
The importance of package size goes without saying. Package size directly affects users’ download, retention, and some manufacturers’ pre-installation requirements must be less than a certain value. However, with the iterative development of the business, the application will become larger and the installation package will continue to expand, so the package size reduction is a long-term governance process.
plan
Alipay has also been working on optimizing package size, and we have introduced many solutions. For example: proGuard code confusion, images from PNG to Tinypng to WebP, introduction of 7ZIP compression scheme, etc. This scheme is different from the above conventional schemes. By directly deleting useless information in dex, the size of Alipay package can be instantly reduced by 2.1m, without affecting the whole operation logic and performance, and even reduce the running memory.
Plan to introduce
-
The introduction
Before going into the details, let’s talk a little bit about debugging logic for the entire Java family. JVM runtime loads.class files. Android invented dalvik and ART to make package sizes more compact and run more efficiently, both of which run.dex files. (Of course, art can also run OAT files. Beyond the scope of this article). Therefore, the information content in the DEX file is exactly the same as that in the class file. The difference is that the dex file duplicates the information in the class file. A DEX contains many class files, and there is a big difference in the structure. Dex is a partition structure. Each block is indexed by offset. Dex = dex = dex = dex = dex The structure of DEX can be represented by the following figure:
Dex file structure is actually very clear, divided into several large blocks, header area, index area, data area, map area. This optimization program optimizes and deletes the debugItems area in the data area.
-
What’s the debugItem for?
The first thing you have to know is what’s in the debugItem? There are two main types of information:
- The parameter variables of the function and all local variables
- DebugItem = debugItem = debugItem = debugItem = debugItem = debugItem = debugItem = debugItem = debugItem = debugItem = debugItem = debugItem The second function is to report crash or actively obtain the call stack, because the virtual machine is really executing the instruction set, the stack report will report the corresponding source file line number of crash, at this time it is through this debugItem to obtain the corresponding line number, can be used to intuitively understand the following screenshot:
The above is a common crash message. The line number in the red box is obtained by looking for the debugItem.
-
How big is debugItem?
In the case of Alipay, the debug package is 4-5m, and the Release package is about 3.5m, accounting for about 5.5% of the dex file size, which is consistent with the official data of Google. Wouldn’t it be tempting to just cut that part out?
-
Can I remove the debugItem directly?
Obviously not. If it is removed, all reported crash information will have no row numbers, and all row numbers will become -1, which will make it impossible to find the north. -keep SourceFile and LineNumberTable are used for this purpose. In order to locate problems, almost all developers keep this configuration. Therefore, the core idea of the solution is to remove the debugItem and get the correct line number when crash is reported. As for IDE debugging, this is easier to solve, we only need to deal with the Release package, not the Debug package.
Plan a
The core idea is also relatively simple, that is, the line number search is offline, so that the corresponding relationship of the line number stored in the App is pulled out in advance and stored in the server. When crash is reported, the line number table is pulled out in advance for reverse solution, so as to solve the problem that the crash information is reported without line number and cannot be located. Although the idea is simple, the implementation is still a bit complicated, and the promotion of the line is also more tortuous. The scheme has been adjusted for several times, and the approximate scheme can be abstracted with the following figure:
As shown in the figure above, there are four cores:
- Change proGuard to use ProGuard to remove debugItem (drop -keep lineNumberTable) and dump a temporary dex before dropping the row number table.
- Modify Dexdump to dump the temporary line number table in dex into a DexpcMapping file (mapping between the line numbers of the instruction set and the line numbers of the source file) and save it to the server.
- The crash handler of Hook App Runtime reports the line number of instruction set during crash to the reverse solution platform.
- The reverse solution platform reports the line number of the instruction set and prepares the DexpcMapping file to reverse solve the correct line number.
The above scheme took more than two weeks, and the whole demo was developed. The other transformation points were not very difficult, but the difficulty was to report the line number of the instruction set. We know that all crashes will eventually have a throwable object, which stores the entire stack information. After repeated reading of the source code and attempts, I found that the line number of the instruction set I wanted was also in this object. It can be illustrated by the following simple graph:
Before printing crash stack information, each throwable calls a JNI method provided by the ART virtual machine and returns an internal object called stackTrace stored in the Throwable object. The stackTrace object contains the call stack of the entire method, including the line number of the instruction set. When the actual stack information is retrieved, the art jNI method is called and the stackTrace method is discarded. The bottom layer inverts the official source line number from the instruction set line number in this stackTrace object. Well, it’s pretty simple. Reflection gets the stackTrace object in the Throwable, gets the line number, and reports it. One thing to note here is that the implementation of each virtual machine is different. First, the name of the internal object is stackTrace, some is backstrace, and the type of the internal object is very different. Some are ints, some are longs, and some are object arrays. X, 5. X, 6. X, 7. X, 7.
Scheme 2
In fact, the above scheme is quite perfect without any compatibility problems. For deletion, proGuard is directly used to obtain the line number of instruction set directly from the Java layer, without the need for various hooks. If only crash reporting is needed, scheme 1 is enough, but it is far from enough in Alipay with many scenarios. Such as:
- Performance, CPU, memory exceptions call stack.
- Java call stack in Native Crash.
All of the above cases involve stack information, and in scenario 1, calling internal stackTrace objects in the throwable via reflection simply doesn’t work and requires a different approach. The initial idea was to try hook Art virtual machine. I searched the source code every day to see the points that could hook, but finally I gave up. One was worried about compatibility problems, and the other was that there were too many hook points, which made me panic. Finally, I tried to modify the dex file directly and reserved a small debugItem so that when the system searched for the line number, the line number of the instruction set was consistent with the line number of the source file. In this way, there was no need to do anything. Any line number reported by monitoring was directly changed into the line number of the instruction set. It can be represented by the following schematic diagram:
As shown above: Originally each method will have a debugInfoItem, each debugInfoItem has an instruction set line number and the mapping relationship between the source file line number, I made the modification is actually very simple, is to delete all the redundant debugInfoItem, Only one debugInfoItem is left, all methods point to the same debugInfoItem, and the instruction set line number in the debugInfoItem is the same as the source file line number, so that no matter what way to look up the line number, get the instruction set line number.
In fact, it is not enough to leave a debugInfoItem. To be compatible with all virtual machine search methods, you need to partition the debugInfoItem, and the debugInfoItem table can not be too large. Encountered a pit is on androidO for dex2OAT optimization, will frequently traverse the debugInfoItem, resulting in AOT compilation is relatively slow, and finally through the debugInfoItem partition to solve.
This scheme is quite thorough, without changing proguard or hook native. However, if we only need to deal with the row number of crash, it is still the first proposal, which has a bit of a big change. In the early stage, WE also studied the file structure of DEX every day, dug out every detail, and dared to change only when we had a great confidence.
summary
At present, the scheme has been officially online in Alipay, after several rounds of external grey verification, or relatively stable. The overall package size of Alipay is reduced by about 2.1m, and the real dex size is reduced by about 3.5m.
Through this section, we have a preliminary understanding of how Alipay improves App running efficiency and quality through package size compression on Android client. There are many technical points that we can’t cover because of space constraints. The corresponding technical kernel is also applied in mPaaS and exported. Welcome to experience it:
Tech.antfin.com/docs/2/4954…
We also look forward to your feedback on the design ideas and specific practices of Android package size compression. Welcome to discuss and communicate with us.
Past reading
The opening | modular and decoupling type development in ant gold mPaaS theorypractice probing depth,
Alipay Mobile Terminal Dynamic Scheme Practice
Analysis of Alipay Client Architecture: A Preliminary Study on iOS Container Framework
Analysis of Alipay Client Architecture: A Preliminary Study on Android Container Framework
Analysis of Alipay Client Architecture: Optimization of Startup Speed of Android Client “Garbage Collection”
Analysis of Alipay App Construction Optimization: Optimizing Android Startup Performance through Package Rearrangement
