test



We know that Android also has a virtual machine modeled after Java, but it’s not called a JVM, it’s called a Dalvik/ART VM and there’s a big difference. The Dalvik/ART VM uses the ClassLoader to load classes and resources, but the Jvm loads the class bytecode through the ClassLoader, while the Dalvik/ART VM loads the dex through the ClassLoader.

Android class loaders are divided into two types,PathClassLoader and DexClassLoader, both of which inherit from BaseDexClassLoader

The PathClassLoader code is located in libcore\ Dalvik \ SRC \main\Java\dalvik\ System \ pathClassLoader.java The DexClassLoader code is located in libcore\ Dalvik \ SRC \main\ Java \ Dalvik \system\ dexClassLoader.java The BaseDexClassLoader code is located in libcore\ Dalvik \ SRC \main\ Java \ Dalvik \system\ basedexClassloader.java

  • PathClassLoader

  • Used to load system classes and application classes

  • DexClassLoader

    Used to load JAR, APK, dex files. Loading JAR and APK is also the final extraction of Dex file inside for loading.

2. Hot repair mechanism

The hot fix is to make use of the order of dexElements. When patch.dex of a patch is placed in the first place of dexElements, when loading a bug class found in patch.dex, the bug class will be directly loaded, and the original bug class may be overwritten


Take a look at the PathClassLoader code

public class PathClassLoader extends BaseDexClassLoader { public PathClassLoader(String dexPath, ClassLoader parent) { super(dexPath, null, null, parent); } public PathClassLoader(String dexPath, String libraryPath, ClassLoader parent) { super(dexPath, null, libraryPath, parent); }}Copy the code

DexClassLoader code

public class DexClassLoader extends BaseDexClassLoader { public DexClassLoader(String dexPath, String optimizedDirectory, String libraryPath, ClassLoader parent) { super(dexPath, new File(optimizedDirectory), libraryPath, parent); }}Copy the code

The two classloaders are two or three lines of code, just calling the constructor of the parent class.

public class BaseDexClassLoader extends ClassLoader { private final DexPathList pathList; public BaseDexClassLoader(String dexPath, File optimizedDirectory, String libraryPath, ClassLoader parent) { super(parent); this.pathList = new DexPathList(this, dexPath, libraryPath, optimizedDirectory); } @Override protected Class<? > findClass(String name) throws ClassNotFoundException { List<Throwable> suppressedExceptions = new ArrayList<Throwable>(); Class c = pathList.findClass(name, suppressedExceptions); if (c == null) { ClassNotFoundException cnfe = new ClassNotFoundException("Didn't find class \"" + name + "\" on path: " + pathList); for (Throwable t : suppressedExceptions) { cnfe.addSuppressed(t); } throw cnfe; } return c; }Copy the code

Create an instance of the DexPathList class in the BaseDexClassLoader constructor. The DexPathList constructor creates a dexElements array

public DexPathList(ClassLoader definingContext, String dexPath, String libraryPath, File optimizedDirectory) { ... this.definingContext = definingContext; ArrayList<IOException> suppressedExceptions = new ArrayList<IOException>(); // Create an array this.dexElements = makeDexElements(splitDexPath(dexPath), optimizedDirectory, suppressedExceptions); . }Copy the code

BaseDexClassLoader then overrides the findClass method and calls pathList.findClass, jumping into the DexPathList class.

/* package */final class DexPathList { ... Suppressed public Class findClass(String name, List<Throwable> suppressed) {// Override this array for (Element Element: DexElements) {// Initialize DexFile DexFile dex = element.dexfile; if (dex ! Class clazz = dex.loadClassBinaryName(name, definingContext,) {// Call DexFile loadClassBinaryName to return Class instance clazz = dex.loadClassBinaryName(name, definingContext, suppressed); if (clazz ! = null) { return clazz; } } } return null; }... }Copy the code

It iterates through the array and initializes DexFile. If DexFile is not empty, it calls the loadClassBinaryName method of the DexFile Class to return the Class instance. The ClassLoader iterates through the array and loads the dex file in the array. The ClassLoader will not load the buggy class after loading the correct class. We will place the correct class in the Dex file, which is placed before the dexElements array.

CLASS_ISPREVERIFIED problem

If static methods, private methods, constructors, etc., are all in the same dex file when the verify option is turned on at virtual machine startup, Then the class is marked with CLASS_ISPREVERIFIED, and once the class is marked with CLASS_ISPREVERIFIED, no other dex can replace the class. Therefore, you must find a way to prevent the class from being marked with the CLASS_ISPREVERIFIED flag.

In order to prevent the class from being marked with CLASS_ISPREVERIFIED, the qZone development team proposed a method to add a prepared hack.dex to the first item of dexElements and make all the classes of subsequent dex reference one of the hack.dex classes. In this way, all the classes in the original class1. Dex, class2. Dex, and class3. Dex reference hack.dex classes, so none of them is marked with the CLASS_ISPREVERIFIED flag.

For example, Qzon team introduced hot patch dynamic repair technology of Android App (this must see!! He is a hotfix veteran and a key source of plagiarism.)

Load class files dynamically and then call reflection to complete the repair principle:

JavaWhen the program is running, the JVM loads the class file into the memory through the ClassLoader. Only when the class file is loaded into the memory can it be referenced by other classes to make the program run correctly.

There are three types of ClassLoaders in Java.

1. Bootstrap ClassLoader 

Written in C++ and started by the JVM.

Rt. jar, resources. Jar, charsets. Jar, and class files in %JRE_HOME/lib/



2.Extension ClassLoader

Java class, inherited from URLClassLoader

Extension class loader, corresponding files are %JRE_HOME/lib/ext jar and class etc



3.App ClassLoader

Java class, inherited from URLClassLoader

System class loader, the corresponding files are all the JARS and classes in the application classpath directory

A note here: Only class files loaded by the same classloader instance and with the same file name are considered the same class.

Here’s a quick example:

If you want to load your own class file, you can customize a classloader.\

How to customize a ClassLoader

Create a new class that inherits from java.lang.ClassLoader and override its findClass method. Convert the class bytecode array to an instance of the class class by calling the loadClass method

I’m going to create a class called Log, very simple, just print

public class Log {  Copy the code
Then I'll create a new MyClassLoader that inherits from ClassLoaderCopy the code


The industry is more famous alibaba AndFix, HotFix(internal test), Dexposed, Qzone super patch and Tencent Tinker(open source) as well as Nuwa, Tencent Bugly,RocooFix and so on

Summary of thermal repair of Dex

At present, there are basically four schemes for thermal repair of Dex:

  • For the Ali system, start with the native layer, see AndFix
  • Qzone solution, pile, see Android App hot patch dynamic repair technology introduction
  • For the solution of wechat, see the Evolution of wechat Android Hot Patch Practice, dexDiff and dexPatch. The method is very exciting and requires full insertion. However, some prematurely loaded classes need to be deleted in the dex of full insertion, otherwise, an exception is also reported on class is pre verified. Another drawback is the combination of memory and built-in storage space. The way of reading on wechat is similar to that on wechat. See the Android Patch solution and continuous delivery. However, reading on wechat is miniloader, which is easy to ANR when started.
  • Meituan’s scheme, namely the scheme of Instant Run, see Android hot Update scheme Robust

In addition, wechat’s solution is multi-classLoader, which can solve the problem that patch cannot take effect on some models by using Multidex, and also brings another advantage. This multi-classloader method uses the code of Instant Run. If there is a native library repair, it will also bring great convenience.

Summary of Native Library hotfix

For r the restoration of Native Libraray, there are basically two schemes at present.

  • Similar to multidex dex, insert directory to the front of the array, specific article see Android hot update so library hot update, need to deal with system compatibility problems, system separation line is Android 6.0
  • The second method needs to rely on multiple classLoaders. When constructing BaseDexClassLoader, obtain the native library of the original Classloader and use the environment variable separator (colon). The patch native Library is connected to the original directory, and the patch directory is in the front, which can also achieve the purpose of repair. The disadvantage is that it needs to rely on the hot repair of DEX, and the advantage is that the application of native Library does not need to deal with compatibility problems. Of course, compatibility issues also need to be dealt with when releasing from patch.

In principle, the above schemes can be simply divided into three categories:

The principle of plan
Native hook plan AndFix
QQ space proposed the Classloader replacement class scheme Nuwa, HotFix, RocooFix
Dex replacement for cold Swap of Instant Run Tinker
Advantages and disadvantages analysis


Test module AndFix This scheme Tinker
Class to replace no yes yes
Resources to replace no no yes
Do YOU need to restart the system? no yes yes
Compatibility stability unstable The best stable


Below, we will introduce qzone super hot patch technology, wechat Tinker and Ali Baichuan HotFix technology respectively.

Qzone super patch technology

The super patch technology is based on the DEX subcontract scheme, using the principle of multi-Dex loading. The general process is: after fixing the BUG method, put it into a separate DEX, insert it at the front of the dexElements array, and let the virtual machine load the fixed method.

 

When patch.dex contains test. class, it will be loaded preferentially. If it encounters test. class in subsequent dex, it will be returned directly without loading, thus achieving the purpose of repair.

 

One problem, however, is that an exception is generated when two classes calling the relationship are not in the same DEX. As we know, when APK is installed, the virtual machine needs to optimize classes.dex into an odex file before it executes. During this process, the verify operation of the class is performed. If all the calling classes are in the same DEX, the “CLASS_ISPREVERIFIED” flag is marked and then the Odex file is written.

 

Therefore, in order to patch the class normally, it is necessary to avoid the class being marked with the ‘CLASS_ISPREVERIFIED’ mark. Specifically, a class is placed separately in another DEX for other classes to call.

 

Let’s reverse the mobile QQ space APK to see the specific implementation:

 

Enter the program entry ‘QZoneRealApplication’ and proceed with two steps in ‘attachBaseContext’ : Fix unexpected DEX problem caused by the ‘CLASS_ISPREVERIFIED’ flag, and load the repaired DEX.

 

 

1. Fix Unexpected DEX Problem

So if I look at the code,

 

As you can see, here is to load a libs directory dalvikhack.jar. Go to assets/libs of your project and unzip it to get ‘classes.dex’.

 

 

Then, the constructor of each class references the unique class AnitLazyLoad in the other DEX so that the class is not marked CLASS_ISPREVERIFIED.

 

In the absence of fixes, set DO_VERIFY_CLASSES to false to improve performance. Set to true only if a fix is needed.

 

 

How to load it in is basically the same as the second step below.

 

2. Load the restored DEX

From loadPatchDex () method to enter, after several times of jump, reach the core code, ` SystemClassLoaderInjector. C ` (). Due to the confusion and multiple method jumps, the core code section is arranged as follows:

 

The repair steps are as follows:

1. Obtain the Classloader of the current application, that is, BaseDexClassloader

2. Obtain his DexPathList attribute object pathList through reflection

3. Convert patch.dex to Element by calling dexElements of pathList through reflection []

4. Merge the two elements [] and put patch.dex first

5. Load Element[] to repair

 

The overall flow chart is as follows:

 

From the flow chart, the characteristics of this approach can be clearly identified:

Advantage:

  1. There is no synthetic package (compared with wechat Tinker), the product is smaller and more flexible
  2. Can achieve class replacement, high compatibility. (Certain Samsung phones don’t work)

Inadequate:

1. It does not take effect immediately and must be restarted to take effect.

2. In order to implement the repair process, two Dex must be added to the application! There is only one class in Dalvikhack. dex, which has little impact on performance. However, for patch.dex, it takes a lot of time to load when a certain number of classes are repaired. An increase in startup time of more than 2s is unacceptable for a carrier class application like Mobile.

3. In ART mode, if the structure of the class changes, memory problems will occur. To solve this problem, it is necessary to load all relevant calling classes, parent classes, and subclasses into patch.dex, which leads to the abnormal size of the patch package and further increases the time consuming when the application is started and loaded.

 

2. Wechat Tinker

Wechat proposed a DEX differential package to replace DEX as a whole in view of the deficiency of qzone super patch technology. The main principle is basically the same as qzone super patch technology, the difference is that patch.dex is not added to elements array, but is given patch.dex by differential method, and then merges patch.dex with classes.dex, and replaces the old dex file as a whole. In order to achieve the purpose of restoration.

 

Let’s reverse the APK of wechat to see the specific implementation:

Find the application entry ‘TinkerApplication’ and call ‘loadTinker()’ on ‘onBaseContextAttached()’,

 

Enter the tryLoad() method of TinkerLoader,

 

From the method name, foreseeable in tryLoadPatchFilesInternal () attempts to load the local patches, again after the jump into the core repair class SystemClassLoaderAdder. In the class.

 

As you can see from the code, specific fixes are taken according to the Android version, but the principle is the same. Let’s take V19 as an example,

 

Reflection calls the makeDexElements() method of PatchList to replace the local dex file directly with the Element[] array.

 

For how to merge patch.dex and classes.dex, wechat starts a new process here, and starts TinkerPatchService of the new process to merge.

 

The overall process is as follows:

 

From the flow chart, it is also obvious to find the characteristics of this method:

Advantage:

  1. Synthesize the entire package without inserting code in the constructor, preventing verify, verify, and opt from being completed at compile time and not at run time.
  2. Improved performance. High compatibility and stability.
  3. Developer transparency, no additional processing of packages is required.

Inadequate:

1. The super patch technology does not take effect immediately. You must restart the application to take effect.

2. You need to start a new process for the application to merge, and the merge fails easily due to memory consumption.

3. The merge occupies extra disk space. For an application with multiple DEX files, if you modify multiple DEX files, you need to deliver multiple patch. DEX to merge the corresponding classes. DEX, which is more serious, and the merge failure rate is higher.

 

3. Alibaichuan HotFix

Compared with qzone super patch technology and wechat Tinker, the HotFix service launched by Alibaichuan is positioned at the scene of urgent BUG repair and can fix bugs in the most timely way. The pull-down patch takes effect immediately without waiting.

 

1. Implementation principle of AndFix

Different from qzone super patch technology and wechat Tinker, which adds or replaces the whole DEX, AndFix provides a way to modify the Filed pointer in Native during runtime to realize the replacement of the method, so as to achieve immediate effect without restart and no performance consumption for the application.

The schematic diagram is as follows:

 

2, AndFix implementation process

 

For the replacement of the implementation method, it needs to be operated in the Native layer through three steps:

 

Next, Dalvik device is taken as an example to analyze the specific implementation process:

 

1, the setup ()

 

 

For Dalvik, following the JIT just-in-time compilation mechanism, the libdvM. so dynamic library needs to be loaded at runtime to obtain the following internal functions:

1) dvmThreadSelf() : query the current thread;

2) dvmDecodeIndirectRef() : Gets the ClassObject based on the current thread.

 

2, setFieldFlag

 

The purpose of this operation is to make private and protected methods and fields public so that they can be seen and recognized by dynamic libraries, which ignore fields and methods that are not public attributes.

 

3, replaceMethod

 

 

This step is the core of method replacement, and the replacement process is as follows:

 

 

AndFix also supports ART devices, and the specific process is similar to Dalvik, which will not be described here.

 

From the technical principle, it is not difficult to see several characteristics of Alibaichuan HotFix:

 

Advantage:

  1. The immediacy of BUG fixes
  2. The PATCH package also adopts the differential technology, and the generated PATCH volume is small
  3. No intrusion to applications, almost no performance loss

Inadequate:

  1. New fields are not supported, the

    method is modified, and resource replacement is not supported.
  2. Due to the manufacturer’s custom ROM, a few models are not supported.

Comprehensive analysis is as follows:

 


  



Pit and solution of thermal repair technology

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

As we can see, the qzone super patch technology and wechat Tinker repair principle are both based on class loading. In terms of functions, they have supported the replacement and addition of classes and resources, with very powerful functions. Since we already have such a powerful hot repair technology, why ali Baichuan launched its own hot repair solution, HotFix?

 

I. Impact of multiple DEX on performance

As we know, the MultiDex solution was originally used to solve the problem of 65K application methods, and now Google has officially supported MultiDex implementation. However, super patch technology and Tinker, as a hot repair scheme, add multiple DEX to the application in life, and the biggest problem of multi-dex technology is the performance pit, so it is no doubt that the patch technology based on this scheme will affect the performance of the application.

1. The startup loading time is too long

As we can see, both the super patch technology and Tinker choose to load the patched dex at the attachBaseContext() of the Application. Even though this is the best time to load the dex, it still brings great performance problems, starting with too long time.

When the application is started, the VM performs the dexopt operation to convert the patch. DEX file to the Odex file, which takes a lot of time. This process, in turn, must be performed synchronously in the main thread, otherwise it cannot be successfully repaired. As for the loading time of DEX, the following time tests have been done.

 

As can be seen from the above table, as the size of patch.dex increases, the startup time increases linearly without any optimization. For an application, this is disastrous.

 

2. ANR and Crash of applications are easily caused

Due to the multiple DEX loads, the startup time is longer, which makes it easier to trigger the ANR of the application. We know that when an application waits on the main thread for more than 5s, it will directly cause a long time of no response and exit. In order to ensure that ART does not have address disorder, super patch technology needs to add all related classes into the patch. However, wechat Tinker adopts a differential package combination loading method, which will make the volume of DEX to be loaded become very large. This also leads to ANR situation to a large extent.

 

In addition to the application of ANR, the multi-DEX mode is also easy to lead to Crash. In order to avoid address confusion in ART devices, it is necessary to add all related classes of the modified class into the patch. There will be a question here. In order to ensure the minimum volume of the patch package, can we ensure the introduction of all related classes without the introduction of irrelevant classes? Once no associated class is introduced, the following exception occurs:

  • NoClassDefFoundError
  • Could Not Find Class
  • Could Not Find Method

These exceptions will directly cause the application Crash exit.

 

Therefore, it is not difficult to see that if we need to fix a BUG that is not Crash, but causes a more serious Crash because we do not add related classes, the gain is more than the loss.

 

In general, the nature of hot fixes is to ensure a more stable application, not to introduce greater risk and instability for greater functionality.

 

Two, hot repair or plug-in?

 

We talk a lot about hot fixes and plugins, which are hot new technologies right now. Before we go on, we need to explain these two concepts.

 

  • Hot fix: An online fix that pushes patches when urgent bugs occur in an online application in order to avoid reissues and ensure timely fixes.
  • Pluginization: An application is divided into parts and loaded into an application as a plug-in, essentially using the same hotfix technique, but with more engineering practices, allowing it to support large-scale code updates and resource and SO package updates.

 

Obviously, conceptually, plug-in usage scenarios are more functional, while hot fixes emphasize minor fixes. In this sense, plug-ins are more powerful and can do more things. From the perspective of the replacement and update of classes and resources, qzone super patch technology and wechat Tinker are not so much hot fixes as practice of plug-in technology.

The qzone super patch technology and wechat Tinker provide more powerful functions, but have a greater impact on the performance and stability of the application. The BUG fixing in this use scenario is not clear enough, and it is too heavy.

 

For the performance loss of the application, we can take an example to make a comparison:

The startup and loading time of an APP is about 3s, which itself is based on the realization of multi-DEX mode.

Access the three hotfix services respectively, and according to the data provided by Tencent and Tinker, the following scenario will emerge:

1. Alibaichuan HotFix: The startup time is almost not increased, and there is no additional disk consumption during the runtime.

2. Qzone super patch technology: if the application has 700 classes, the startup time will increase by more than 2.5s to more than 5.5s.

3. Wechat Tinker: Suppose that the application has five DEX files, and the five DEX files are modified respectively to produce five patch. DEX files, and five patch merges are required. Assuming that each patch is 1M, 7.5M of disk space will be occupied.

Obviously, alibaichuan HotFix is more suitable for the scenario of fixing urgent bugs. It is much lighter, can take effect without restarting, and has little impact on performance.

Found a lot of information and read a variety of article source code to write this article, a lot of local understanding is not so in-depth, a lot of things are also picked up by others, I hope you criticize.

Since the reference: