The background,
With the rapid development and continuous iteration of the business, the package size of toy dezhi APP is also increasing. In only four months, it has increased from 127.4M in V3.0.2 to 174.5M in V3.5.0, an increase of about 37%. It is conceivable that if not controlled in time, the package size will soon exceed 200M.
If the installation package is too large, the download conversion rate will be affected. Statistics released at Google’s Developer conference show:
For every 6MB increase in package size, the app download conversion rate drops by 1%,
The average download conversion rate increases by 0.5-1.5% each time the package size decreases by 10MB.
There are two concepts of installation package size: download size and installation size.
Download size: the size of the compressed App downloaded over the network. To save traffic, users download compressed packages, and the decompression process is called installation.
Installation size: the disk space occupied on the user’s device after the App is decompressed. This is the size you see on the App Store. The larger the install size is, the more likely it is to affect the user’s willingness to download.
If the download size is too large, Apple will restrict users from downloading apps over cellular networks.
- In September 2017, after iOS 11, the download limit was increased from 100 MB to 150 MB
- In May 2019, the download limit was increased from 150 MB to 200 MB
- In September 2019, after iOS 13, users can choose whether to use a cellular network to download the device if the download size exceeds 200 MB. However, the cellular network will still not be available for iOS 13 and below
Although Apple is gradually loosening its restrictions. However, if the download size exceeds 200 MB, it will definitely have a great impact on APP download cost and promotion efficiency.
However, if the installation size is too large, it will affect the retention rate of users. After all, when users run out of mobile phone memory, they will preferentially delete apps that occupy a large amount of memory.
So reducing the download size and installation size is our goal.
2. Package size analysis
By unpacking an IPA file, we can see that an.app file consists of three main parts:
- Resource files: mainly pictures, audio, video, and other resources.
- Executable file: The body of a program, which is the file generated by compiling links to our code, static libraries, and dynamic libraries.
- Bundle: A third-party or resource bundle used in a project.
However, the size of an app is not exactly the size of the package. After the app is uploaded to AppStore Connect, Apple will also do some processing on the installation package, and the change of the test installation package cannot correspond to the change of the real download size. The processing mainly includes:
- App Slicing is the icing of different architectures.
- Asset-car images leave only the specific size and compression algorithm variations required by the device;
- __TEXT segment encryption;
This is part of the reason for the different package sizes you see on different devices.
Through the analysis, it can be seen that the way of slimming is mainly for the optimization of executable files and resources.
Third, executable file optimization
1. Delete useless classes
General useless code screening can be divided into dynamic and static two ways. The static approach involves scanning code, participating in the build process, or analyzing the end product to determine which code is not being used. The dynamic approach relies mainly on pegs or runtime information to get what code is not executing.
1.1 Dynamic Search
Line-level code coverage based on staking:
The staking scheme based on GCOV or LLVM Profile binary can collect the staking data at run time to guide the removal of useless code. However, the limitations of pile insertion scheme are also obvious. Pile insertion will degrade the size and performance of binary itself, and the original pile insertion scheme is unable to pass the review line. Data collection can only be limited offline.
A runtime-based lightweight Runtime “class coverage” solution:
When +initialize is invoked for the first time in an Objc class, the system will automatically flag that it has been invoked, and the state will be stored at bit 29 in the Flags field of data in metaClass. This can be obtained using flags & RW_INITIALIZED.
1.2 Static Search
In the Mach-o file, __DATA ‘ ‘__objc_classrefs records the address of the referenced class, __DATA“ “__objc_classlist records the address of all classes, we print the corresponding information through otool, and then take the difference value of the two, and then symbolize. You get information about classes that are not referenced.
- through
otool -v -s __DATA __objc_classrefsGets the address of the reference class (explicitly used). - through
otool -v -s __DATA __objc_classlistGets the addresses of all classes.
- Subtracting the reference class from all the class information, we get the address information of the unused class.
- through
nm -nmCommand to get the address and corresponding class name.
Run otool -v -s __DATA __objc_classrefs to obtain the address of the referenced class.
def classref_pointers(path, binary_file_arch): ref_pointers = set() lines = os.popen('/usr/bin/otool -v -s __DATA __objc_classrefs %s' % path).readlines() for line in lines: pointers = pointers_from_binary(line, Binary_file_arch) ref_pointers = ref_pointers. Union (Pointers) return ref_pointers copy the codeCopy the codeRun otool -v -s __DATA __objc_classlist to obtain the addresses of all classes.
def classlist_pointers(path, binary_file_arch): list_pointers = set() lines = os.popen('/usr/bin/otool -v -s __DATA __objc_classlist %s' % path).readlines() for line in lines: pointers = pointers_from_binary(line, Binary_file_arch) list_pointers = list_pointers. Union (Pointers) return list_pointers copy codeCopy the codeSubtracting the reference class from all the class information, we get the address information of the unused class.
Unref_pointers = classlist_pointers(path, binary_file_arch) - classref_pointers(path, binary_file_arch) copies the codeCopy the codeThe address and corresponding class name can be obtained by using the nm-nm command.
def class_symbols(path): symbols = {} re_class_name = re.compile('(\w{16}) .* _OBJC_CLASS_$_(.+)') lines = os.popen('nm -nm %s' % path).readlines() for line in lines: result = re_class_name.findall(line) if result: (address, symbol) = result[0] symbols[address] = symbol return Symbols copy codeCopy the codeThe result is output to TXT
Because it is a static lookup, dynamically generated classes, such as those generated by reflection, are considered unreferenced, so you need to manually check the list after finding it.
Optimization result: 110 useless classes were deleted, and the profit was 0.5m.
2. Optimize compilation options
2.1 open LTO
Compilation option link-time Optimization
According to Apple, enabling LTO will increase the running speed under Release by 10% and reduce the package size.
Apple uses LTO extensively internally
- Typically 10% faster than executables from regular Release builds Multiplies
- with Profile Guided Optimization (PGO)
- Reduces code size when optimizing for size
However, the drawback is that debug compilation is much slower, and the second compilation will be complete, so we only enable LTO in Release mode.
2.2 Optimization Level
Optimization Level refers to the compiler Optimization Level used by clang. Clang-code Generation Options can be found in the documentation of clang.
-o0 Means “no optimization” : this level compiles the fastest and generates the most debuggable code.
-O1 Somewhere between -O0 and -O2.
-O2Moderate level of optimization which enables most optimizations.
-O3 Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in an attempt to make the program run faster).
-Ofast Enables all the optimizations from -O3 along with other aggressive optimizations that may violate strict compliance with language standards.
-Os Like -O2 with extra optimizations to reduce code size.
-Oz Like -Os (and thus -O2), but reduces code size further.
Xcode defaults to -o0 for debug and -OS for release. After testing, using -oz will reduce the package size by 3M or so, but crash will appear in some pages, which is found to be memory problems caused by delayed release. For security reasons, the optimization level of -OS is currently used.
2.3 Symbol Correlation
Symbols refers to all the variables, classes, functions, enumerations, variables and address mapping in the program, as well as some debugging symbols used to locate the code in the source code. Symbols have a very important relationship with the location of breakpoints and stack symbolization.
2.3.1 Strip Linked Product (STRIP_INSTALLED_PRODUCT)
If enabled, the linked product of the build will be stripped of symbols when performing deployment postprocessing.
If set to yes, the Symbols will be clipped when packaging.
Not all symbols are required, such as Debug maps, so Xcode gives us Strip Linked Products to remove unwanted symbol information (symbols corresponding to the option selected in Strip Style), After removing the symbolic Information, we can only use dSYM for symbolization, so we need to change the Debug Information Format to DWARF with dSYM file.
2.3.2 Strip Debug Symbols During Copy (COPY_PHASE_STRIP)
Specifies whether binary files that are copied during the build, such as in a Copy Bundle Resources or Copy Files build phase, It does not cause the linked product of a target to be stripped — use Strip Linked Product (STRIP_INSTALLED_PRODUCT) for that.
Similar to Strip Linked Product, but this removes tripartite libraries, resources, or Extension Debug symbols copied into the project package, also using the Strip command. This option has no preconditions, so we only need to enable it in Release mode, otherwise we won’t be able to debug breakpoints and symbolize tripartite libraries.
2.3.3 Symbols Hidden by Default (GCC_SYMBOLS_PRIVATE_EXTERN)
When enabled, all symbols are declared private extern unless explicitly marked to be exported using attribute((visibility(“default”))) in code. If not enabled, all symbols are exported unless explicitly marked as private extern.
If this is set to yes, all symbols are declared private extern. After testing, this can actually reduce the package size.
The Settings in the project are as follows:
target.build_configurations.each do |config|
config.build_settings['COPY_PHASE_STRIP'] = 'YES'
config.build_settings['GCC_SYMBOLS_PRIVATE_EXTERN'] = 'YES'
config.build_settings['STRIP_INSTALLED_PRODUCT'] = 'YES'
end
Copy the codeCompiler option optimization result: yield 4.2m
3, __TEXT segment migration
The executable file of iOS is a MachO file. MachO structure is mainly divided into Header, Load Commands and Data.
HeaderContains general information about the binary, byte order, schema type, number of load instructions, and so on. This allows you to quickly confirm information such as whether the current file is 32-bit or 64-bit, the corresponding processor, and the file type.Load CommandsIt’s a table with a lot of content. The content includes the location of the region, symbol table, dynamic symbol table, etc. They describeDataLayout information in binary files and virtual memory, with this layout information can be knownDataHow they are arranged in binary files and virtual memory.
DataStores the actual content, usually the largest part of the object file, containing seINTERFACES specific data, such as static C strings, OC methods with/without parameters, and C functions with/without parameters.
Here is the structure to view in MachOView:
The structure of Data can be divided into multiple segments, mainly __PAGEZERO, __TEXT, __DATA, __LINKEDIT:
__PAGEZEROIt’s in the executable, it’s not in the dynamic library. This segment starts at address 0 (where the NULL pointer points to), and is an unreadable, unwritable, and unexecutable space that can throw exceptions if the NULL pointer accesses it.__TEXTA code segment in which code is stored. The segment is readable and executable, but not writable.
__DATAA data segment, which stores data, is readable and writable but not executable.__LINKEDITThe section is used to store signature information. The section can only be read, but cannot be written or executed.
Each Segment can be divided into one or more sections, and __TEXT is a Segment of Data.
__TEXT segment migration:
A Mach-O file consists of four stages: preprocessing -> compiling -> assembling -> linking.
We can move sections during link time by adding arguments to Other Linker Flags.
-Wl,-rename_section,__TEXT,__cstring,__RODATA,__cstring
-Wl,-segport,__RODATA,rx,rx
Where -wl tells Xcode that the parameters behind it are added to the Ld linker and will take effect during the link phase.
The first line creates a new __RODATA section and moves __TEXT and __cstring to __RODATA and __cstring.
The second line of arguments grants __RODATA readable and executable permissions.
Let’s look at the mach-o file before moving __TEXT,__cstring:
After the build is complete, take a look at the mach-o file after moving __TEXT,__cstring:
This successfully moves some sections in the __TEXT Section.
Facebook used this approach early on to solve the __TEXT segment size limitation problem, see Analysis of the Facebook.app for iOS
Facebook avoids this limitation by moving some if the __TEXT sections into the read only __RODATA segment. Implementing this trick is really simple: you just need to add a linker flag to rename the chosen sections. And it appears you need absolutely nothing at runtime: the renamed sections will be found automatically. This linker flag is described in the ld man page:
-rename_section orgSegment orgSection newSegment newSection
Renames section orgSegment/orgSection to newSegment/newSection.
You could use it to rename the (__TEXT, __cstring) section to (__RODATA, __cstring) by simply adding this line into the Other Linker Flags (OTHER_LDFLAGS):
-Wl,-rename_section,__TEXT,__cstring,__RODATA,__cstring
Copy the codeToutiao did the same when it reduced the download size,
You can do this by adding the following parameters to Other Linker Flags
The function is to move the __TEXT section to another section and grant read and execute permissions.
So why does __TEXT segment migration reduce download sizes?
The reason is that after uploading an App to App Store Connect, Apple encrypts it and compresses it into IPA. Encryption has little effect on the size of the executable itself, but greatly affects compression efficiency. The __TEXT Segment is the most important part of the encryption Segment. The encryption range can be reduced by reducing the __TEXT Segment, so some sections in the __TEXT Segment can be migrated to other segments.
Optimization results: The installation size is reduced by 0.2m, and the download size is reduced by 25M.
4. Tripartite library correlation
1. Promote the use of the simplified version of the live broadcast SDK. Since the live broadcast scene does not need the ability of real-time audio and video, super player SDK and AI special effects components, the live broadcast SDK is modified. In order to prevent problems with the replacement of SDK, gray scale observation of two versions was carried out, and the replacement was confirmed after full testing.
2. Push some SDKS to be removed. Removed some SDKS that could be replaced.
Optimization result: revenue of 7M.
Fourth, resource optimization
Resource optimization mainly refers to the optimization of image resources and other JSON, audio, video and other resources.
1. PNG image compression
Two schemes for PNG compression are compared:
TinyPNG
Lossy compression, primarily using Quantization, supports almost perfect transparency by combining similar colors in images, compressing 24-bit PNG images into much smaller 8-bit color values, and removing unnecessary metadata.
Website: tinypng.com/
ImageOptim
Lossless compression, image files often contain some comments, color Profile and other redundant information, removed image quality unchanged, smaller and faster loading. ImageOptim compresses the image in this way, first analyzes the image, finds the optimal compression parameters, removes irrelevant information and reduces the volume.
Website: imageoptim.com/mac
After compression test, it is found that TinyPNG compression effect is much better than ImageOptim, TinyPNG compression ratio is about 65%, ImageOptim compression ratio is about 30%, and the naked eye looks no difference.
In the process of using, we found that although some PNG images became smaller after compression, the change was not obvious after packaging, and some even became larger. According to the analysis of PNG images in IPA and reference, Apple also compresses PNG images. This compression process is to speed up the processing of images and convert them into CgBI format, which is more convenient for processing. In addition, the IDAT data block that stores the actual image data is modified, and the Filter method and zlib compression method that determine the size of THE IDAT data block are changed. Since CgBI IDAT is in BGRA format, no matter whether the PREVIOUS IDAT has Alpha channel, Alpha channel will be added during processing. Secondly, because the filter of each row of data is different, Apple uses the same filter for each row of data by default. The original file can use different filters for different data lines through better algorithms to provide easier data compression for subsequent data compression. Therefore, Apple’s optimization of PNG may result in some PNG images becoming larger.
Therefore, we will also convert partial compressed PNG images into WebpP for further processing.
Optimization result: revenue 5M
2. Convert PNG images into WebP images
Compared to PNG format, WebP has a better image data compression algorithm, resulting in a smaller image size. So some larger images will be converted to WebP images.
Cwebp — Compress an image file to a WebP file
Installation method: Brew Install webp
Usage:
cwebp [options] -q quality input.png -o output.webp
-loss(lossy compression, default) -lossless(lossless compression)
-q: quality index (compression ratio). Lossy compression is effective and lossless compression is ignored
Input. PNG: image to be converted
-o: Enter the image name format
# PNG into webp toWebp () {filePath = echo $1 | sed 's / / / / g' fileName = {fileName# # * /} fileName = echo $fileName | sed 's / / _ / g' If [[-e $LOCAL_CWEBP_PATH]]; if [-e $LOCAL_CWEBP_PATH]]; then cwebp -quiet "$filePath" -o $newFilePath$fileName.webp else $basedir/bin/cwebp -quiet "$filePath" -o $newFilePath$fileName.webp fi echo $filePath printResult $? "${filePath# # * /} ☑ $newFilePath $fileName. Webp"}Copy the codeDuring the conversion, WE thought that the script would convert the PNG image to webP image, and then hook image loading mode could read the WebP image. However, we found that the PNG image is managed by imageset, and the image name used in the code may be inconsistent with the PNG image name.
Solution: Scripts converted to WebP do not use the PNG image name directly, but use the imageset name that manages the PNG.
After solving the problem, we found another problem after running. The images were enlarged. After investigation, we found that PNG has three kinds of 1x, 2X and 3X, and a 60×60 pixel 3X image generates UIImage object with 3 scale and 20×20 size. UIImage has a scale of 1 and a size of 60, so the image gets bigger when it’s displayed. Fortunately, a conversion method to modify the scale parameter is found in SDImageCoder.
/** Decode the image data to image. @note This protocol may supports decode animated image frames. You can use `+[SDImageCoderHelper animatedImageWithFrames:]` to produce an animated image with frames. @param data The image data to be decoded @param options A dictionary containing any decoding options. Pass @{SDImageCoderDecodeScaleFactor: @} (1.0) to specify the scale factor for the image. Pass @ {SDImageCoderDecodeFirstFrameOnly: @(YES)} to decode the first frame only. @return The decoded image from data */ - (nullable UIImage *)decodedImageWithData:(nullable NSData *)data options:(nullable SDImageCoderOptions *)options;Copy the codeHowever, another problem appeared again. I wanted to convert the 3X PNG image to WEBP and then transfer the scale parameter to 3. However, due to the irregular image management, some PNG images only have 1x image and some only have 2X or 3X image, so we need to convert them according to what kind of PNG image webP is. Pass the corresponding scale parameter.
Then, it was found that some images could not be loaded in the later test. The investigation found that these images were read in XIB, and the METHOD of XIB reading PNG was not through imageNamed. Then, the first idea was to hook XiB to read PNG, but Apple did not expose the method of XIB loading PNG. There is also some information that can be through hook UINibDecoder decodeObjectFotKey method, but feel that is not very rigorous, so THE USE of PNG in XIB, I used another way: in the code will control using imageNamed method to read again the picture.
It is also important to note that some PNG images that support region stretching will be distorted when converted to WebP. This part of the diagram is not suitable for webP conversion.
Optimization result: revenue 6M
3. Modify the picture management mode in the component library
Asset Catalog is an image resource management method provided by Xcode. Each Asset represents an image resource, but can correspond to one or more PNG images. For example, it can provide @1x, @2x, and @3x images.
The images in the Asset Catalog are compressed during compilation. Then, when the App runs, the API can dynamically select the corresponding real image rendering according to the device scale factor. The images managed using the Asset Catalog will generate an assets.car file in the IPA package.
App Thing is a solution for optimizing the size of App package download resources on the Apple platform. After the App package is submitted and uploaded to the App Store, apple background server will simplify the App package for different devices according to their scale factor, so that different devices need different capacities to download from the App Store. Device 3X does not need to download 1x and 2X maps at the same time.
However, this mechanism is directly based on the Asset Catalog, which means that only images introduced in the Asset Catalog can enjoy App Stochastic. Scattered images copied directly into the App Bundle will still be downloaded on all devices.
Therefore, maximizing Asset Catalog utilization is a big package size optimization point.
So when using Cocoapods for component library management, PNG images in the component library are also managed using The Asset Catalog.
There is another difference in the way resources are imported: There are two methods of importing resources in POD, resource_bundles and resources.
With resources, it’s imported in the main bundle. This way to read the image does not need to change the read mode.
S.resources = ['ResourcesTest/Assets/*.xcassets'] copies the codeCopy the codeUsing resource_bundles, a custom bundle is generated in the main bundle where the resources are stored. Resources need to be read from the corresponding bundle. This approach avoids naming conflicts.
S.r esource_bundles = {' ResourcesBubdlesTest '= > [' ResourcesBubdlesTest/Assets / *. Xcassets]} copy codeCopy the codeIn the process of modification, there are some unexpected gains. It was found that there are problems in the way of importing resources into some component libraries in the project. At the same time, two methods of resource_bundles and resources were specified. This results in images being stored in both the main bundle and the bundle generated by resource_bundles.
The use of resource_Bundles + Asset Catalog is recommended for managing PNG images in the component library.
Optimization result: yield 4.5m
4. Delete useless PNG images
Screening by tool: LSUnusedResources
Optimization result: 56 pictures were deleted and the profit was 1.2m
5. Compress text files
There are some JSON files of Lottie animation stored locally in plaything Dezhi APP, and some optimization effects have been achieved through the packaging and compression of these files.
- Compress local JSON files together into a ZIP;
- When starting, unzip the asynchronous thread and store it in the sandbox.
- The runtime reads JSON from the sandbox;
You can do this for more resource types, such as audio and video.
Optimization result: revenue 1.2m
5. Monitor package size
To control increments, we also monitor package sizes for each version.
For executable file changes, we used LinkMap to analyze and record the size changes of each component.
For resource changes, we will also analyze and record resource size changes from each version of IPA package.
Later, we plan to bayonet of increment, pangu packing platform for automatic package volume analysis, before each branch is merged into the main branch, can reflect the increment size, so that we can make development to develop their own code has a more intuitive feeling, strengthen the development of thin body consciousness in the normal course of encoding, consciously, code for resources to clean up. Strive for zero increment in package volume.
Six, effects,
All the above schemes have been practiced and implemented in Plaything DeZhi APP. After a series of optimization, the overall benefits of plaything Dezhi APP package volume are as follows:
Download size decreased from 136.2m to 78.6m, reduced by 57.6m
The installation size decreased from 174.5M to 140M, decreasing by 34.5M
Download size is most directly reflected in the length of download time. Here is a comparison of the download and installation time of 3.5.0 and 3.6.7 versions:
Before optimization: It takes 64 seconds to download and install
Optimized: The download and installation duration is 43 seconds
Download time reduced by 32.8%
The following are recorded changes in download size and install size for each version since optimization:
Green mountains do not change, green water flow, thank you for your support, I hope this article can help you!!