One, a brief introduction

Hot repair is undoubtedly a hot new technology in the past two years. It is one of the skills that android engineers must learn. Before there were hot fix in an online app if there is a bug, even a very small bug, not timely update possible risks, if to update the app to repackage published to the application market, allow users to download again, thus greatly reduce the user experience, when hot fix, Such questions are no longer a problem.

At present, hot thermal repair schemes can be roughly divided into two groups, which are:

  1. Ali: DeXposed, andfix: start from the bottom binary (C language).

  2. Tinker: Start with Java loading mechanism.

The topic of this article is not about the use of the above two solutions, but about the principles and implementation of hot fixes based on the Java loading mechanism. (Like Tinker, but tinker is not so simple.)

How to fix bugs dynamically in Android

About the concept of bugs, I think there are generally two kinds of bugs (may not be accurate) :

  • The code does not function as expected by the project, that is, the code logic is faulty.

  • The application code is not robust enough to cause the App to crash at runtime.

In both cases, there is usually a problem with one or more classes. In an ideal state, we just need to update the repaired classes to the app on the user’s phone to fix the bugs. But speaking of simplicity, how do you dynamically update these classes? In fact, no matter what kind of heat repair solution, there are definitely several steps:

  1. Deliver a patch (containing the repaired class) to the user’s phone, i.e. let the app download from the server (network transfer)

  2. App uses **” somehow “** to make the class in the patch called by app (local update)

The **” somehow “** here, for this article, is to use the Android class loader, which loads the fixed classes and overwrites the class that should have the problem, theoretically fixing the bug. So, let’s get to know and analyze Android class loaders.

Class loaders in Android

Android has a long history with Java. Jvm-based Java applications use a ClassLoader to load the classes in the application, but we know that Android has optimized the JVM, uses Dalvik, and the class files are packaged into a dex file. The class loaders of the underlying VMS are different. On Android, to load the class files in the dex file, two Android-based class loaders, PathClassLoader or DexClassLoader, are required.

1. View the source code

The general source code can be found in Android Studio, but the source code of PathClassLoader and DexClassLoader is system level source, so it cannot be directly viewed in Android Studio. However, there are two ways to the external view: the first is a view by means of image download the Android source code, but general image source volume is larger, download, and just to see the source of 3, 4, a file on every download source, 3, 4 g does not too smart, so we usually adopts the second way: Go to androidxref.com to view directly, the following will be listed after the analysis of several classes of source address, for the convenience of visitors to browse.

Here is some of the source code for Android 5.0:

  • PathClassLoader.java

  • DexClassLoader.java

  • BaseDexClassLoader.java

  • DexPathList.java

2. Differences between PathClassLoader and DexClassLoader

1) Usage scenarios

  • PathClassLoader: loads only apK files (/data/app directory) that have been installed in the Android operating system. It is the default class loader used by Android.

  • DexClassLoader: can load dex/jar/ APk /zip files in any directory, more flexible than PathClassLoader, is the focus of hot repair.

2) Code differences

Because the source code for PathClassLoader and DexClassLoader is very simple, I will directly copy their full source code:

// PathClassLoader public class PathClassLoader extends BaseDexClassLoader { public PathClassLoader(String dexPath, ClassLoader parent) { super(dexPath, null, null, parent); } public PathClassLoader(String dexPath, String librarySearchPath, ClassLoader parent) { super(dexPath, null, librarySearchPath, parent); } } // DexClassLoader public class DexClassLoader extends BaseDexClassLoader { public DexClassLoader(String dexPath, String optimizedDirectory, String librarySearchPath, ClassLoader parent) { super(dexPath, new File(optimizedDirectory), librarySearchPath, parent); }}Copy the code

Through comparison, two conclusions can be drawn:

  • Both PathClassLoader and DexClassLoader inherit from BaseDexClassLoader.

  • Both PathClassLoader and DexClassLoader call the constructor of the parent class in the constructor, but DexClassLoader passes an additional optimizedDirectory.

3, BaseDexClassLoader

By observing the source code of PathClassLoader and DexClassLoader, we can determine that the truly meaningful processing logic must be in BaseDexClassLoader, so the following focuses on the analysis of BaseDexClassLoader source code.

1) Constructor

Let’s start by looking at what the BaseDexClassLoader constructor does:

public class BaseDexClassLoader extends ClassLoader { ... public BaseDexClassLoader(String dexPath, File optimizedDirectory, String libraryPath, ClassLoader parent){ super(parent); this.pathList = new DexPathList(this, dexPath, libraryPath, optimizedDirectory); }... }Copy the code

  • DexPath: the directory where the program file to be loaded (dex file or JAR/APk /zip file) is located.

  • OptimizedDirectory: output directory of dex files (this directory is specially used to store decompressed program files in jar/ APk /zip formats).

  • LibraryPath: libraryPath used when loading program files.

  • Parent: parent loader

The concept of “application files” is my own definition, because from the point of view of a full App, application files specify the classes.dex file in the APK package; But from a hotfix perspective, program files refer to patches.

Because PathClassLoader only loads dex files in installed packages, DexClassLoader can load not only dex files, but also dex files in JAR, APK, and ZIP files. We know that JAR, APK, and ZIP are actually some compressed formats. To get the dex file in the zip package, you need to decompress it, so DexClassLoader specifies a directory to decompress when calling the parent constructor.

The BaseDexClassLoader constructor logic has changed since Android 8.0. The optimizedDirectory is outdated and no longer takes effect

2) Get the class

The class loader must provide a method for the outside world to find the class to which it is loaded. That method is findClass(), but neither PathClassLoader nor DexClassLoader source code overrides the findClass() method of the parent class. FindClass () does not override findClass(); findClass() does not override findClass();

private final DexPathList pathList; @Override protected Class<? > findClass(String name) throws ClassNotFoundException { List<Throwable> suppressedExceptions = new ArrayList<Throwable>(); FindClass c = pathList.findClass(name, suppressedExceptions);if (c == null) {
        ClassNotFoundException cnfe = new ClassNotFoundException("Didn't find class \"" + name + "\" on path: " + pathList);
        for (Throwable t : suppressedExceptions) {
            cnfe.addSuppressed(t);
        }
        throw cnfe;
    }
    return c;
}

Copy the code

As you can see, the findClass() method of the BaseDexClassLoader actually gets the class from the findClass() method of the DexPathList object (pathList), The DexPathList object happens to have been created in the BaseDexClassLoader constructor. So, let’s look at what’s going on in the DexPathList class.

4, DexPathList

Before analyzing a source code with a lot of code, what do we need to know from the source code? In order not to get lost in the “code sea”, I set two small goals, respectively:

  • What does the DexPathList constructor do?

  • How does the findClass() method of DexPathList get the class?

Why these two goals? Because in BaseDexClassLoader source code mainly used the DexPathList constructor and findClass() method.

1) Constructor

private final Element[] dexElements; public DexPathList(ClassLoader definingContext, String dexPath, String libraryPath, File optimizedDirectory) { ... this.definingContext = definingContext; this.dexElements = makeDexElements(splitDexPath(dexPath), optimizedDirectory,suppressedExceptions); . }Copy the code

In this constructor, the current classloader definingContext is saved and makeDexElements() is called to get the Element collection.

By tracing the source code of splitDexPath(dexPath), we found that the function of this method is actually to convert all program files in the dexPath directory into a File set. And also found that dexPath is a colon (” : “) as a delimiter joined the multiple program files directory string (such as: / data/dexdir1: / data/dexdir2:…). .

The next step is to analyze the makeDexElements() method. Since this section of code is longer, I’ll post the key code and comment it out:

private static Element[] makeDexElements(ArrayList<File> files, File optimizedDirectory, ArrayList<IOException> suppressedExceptions) { // 1. Elements = new ArrayList<Element>(); // 2. Run through all dex files (jar, APk, or zip files)for(File file : files) { ZipFile zip = null; DexFile dex = null; String name = file.getName(); . // If it is a dex fileif(name.endsWith(DEX_SUFFIX)) { dex = loadDexFile(file, optimizedDirectory); // If it is an APk, JAR, zip file (this part is handled slightly differently in different Android versions)}else{ zip = file; dex = loadDexFile(file, optimizedDirectory); }... // 3. Wrap the dex file or zip file as an Element object and add it to the Element collectionif((zip ! = null) || (dex ! = null)) { elements.add(new Element(file,false, zip, dex)); }} // 4. Return the Element set as an Element arrayreturn elements.toArray(new Element[elements.size()]);
}

Copy the code

In this method, you can see a few things. In general, the constructor of DexPathList is to wrap program files (maybe dex, APK, JAR, zip) into Element objects and add them to the Element collection.

In fact, Android class loaders (either PathClassLoader or DexClassLoader) only recognize dex files in the end, and loadDexFile() is the core method of loading dex files. Dex can be extracted from JAR, APK, zip, etc. But I won’t analyze it here, because the first goal is done, so I’ll analyze it later.

2) the findClass ()

Look again at the DexPathList findClass() method:

public Class findClass(String name, List<Throwable> suppressed) {
    for(Element Element: dexElements) {// Iterate through a dex file.if(dex ! Class clazz = dex.loadClassBinaryName(name, definingContext, suppressed);if(clazz ! = null) {returnclazz; }}}if(dexElementsSuppressedExceptions ! = null) { suppressed.addAll(Arrays.asList(dexElementsSuppressedExceptions)); }return null;
}

Copy the code

The DexPathList findClass() method simply iterates through the Element array. If it finds a class with the same class name as name, it returns the class directly. If it does not find a class with the same name, it returns null.

Why call DexFile loadClassBinaryName() to load the class? This is because an Element object corresponds to a dex file, and a dex file contains multiple classes. The Element array contains dex files, not class files. This can be seen from the source code of the Element class and the internal structure of the dex file.

Four, the realization principle of thermal repair

Finally enter the topic, after the analysis of PathClassLoader, DexClassLoader, BaseDexClassLoader, DexPathList, we know, When loading a class on Android, the class loader first obtains (Element[] dexElements) from the Element array in its DexPathList object into the corresponding class, and then loads it. Array traversal is used, but notice that traversal is a dex file.

In the for loop, we first iterate through the dex file and then retrieve the class from the dex file, so we simply pack the repaired class into a dex file and place it at the first Element of the Element array to ensure that the newly repaired class is retrieved (of course, A buggy class exists, but it is placed in the last Element of the Element array, so it has no chance of being retrieved.

Five, the simple implementation of hot repair

With all the theory behind it, it’s time to put it into practice.

1. Obtain the patch in DEX format

1) Fix the problem Java files

In this step, modify the code according to the actual situation of the bug.

2) Compile Java files into class files

After fixing the bug, you can compile the code using Android Studio’s Rebuild Project feature and find the corresponding class file in the Build directory.

Copy the repaired class file somewhere else, such as the dex folder on your desktop. Note that when you copy this class file, you need to copy the full package directory in which it resides. Hypothesis above repair good class files is SimpleHotFixBugTest. Class, then copy the directory structure is:

3) Package the class file as a dex file

A. Dx instruction program

To package a class file into a dex file, you need the dx directive, which is similar to a Java directive. We know that Java directives include javac, JAR, etc. The reason why we can use these kinds of directives is because we have installed JDK, JDK provides Java directives for us. Similarly, dx directives also need to be provided by programs. It’s in the build-tools directory of the Android SDK for each Android version.

B. Use of dx instruction

The dx directive can be used under the same conditions as the Java directive. There are two options:

  • Configure the environment variables (added to the CLASspath), and then the command line window (terminal) can be used anywhere.

  • Without environment variables, directly use the command line window (terminal) in the build-tools/ Android version directory.

The first way is to refer to Java environment variable configuration, and I’m going to use the second way. The command we need to use is:

Dx –dex –output=dex File complete path (space) Directory where the complete class file is to be packed, for example:

dx –dex –output=C:\Users\Administrator\Desktop\dex\classes2.dex C:\Users\Administrator\Desktop\dex

See the following figure for specific operations:

In the blank area of the folder directory, hold down Shift + right click to appear “Open command line window here”.

2. Load the dex patch

Based on the principle, we can make a simple utility class:

*/ public class FixDexUtils {private static final String DEX_SUFFIX =".dex";
    private static final String APK_SUFFIX = ".apk";
    private static final String JAR_SUFFIX = ".jar";
    private static final String ZIP_SUFFIX = ".zip";
    public static final String DEX_DIR = "odex";
    private static final String OPTIMIZE_DEX_DIR = "optimize_dex"; private static HashSet<File> loadedDex = new HashSet<>(); static { loadedDex.clear(); } /** * load the patch using the default directory: Data /data/ package name /files/odex * * @param context */ public static void loadFixedDex(context context) {loadFixedDex(context, null); } public static void loadFixedDex(context context, loadFixedDex) File patchFilesDir) {if (context == null) {
            return; } dex File fileDir = patchFilesDir! = null ? patchFilesDir : new File(context.getFilesDir(), DEX_DIR); // data/data/ package name /files/odex (this can be any position) File[] listFiles = filedir.listfiles (); // data/data/ package name /files/odex (this can be any position) File[] listFiles = filedir.listfiles ();for (File file : listFiles) {
            if (file.getName().startsWith("classes") && (file.getName().endsWith(DEX_SUFFIX) || file.getName().endsWith(APK_SUFFIX) || file.getName().endsWith(JAR_SUFFIX) || file.getName().endsWith(ZIP_SUFFIX))) { loadedDex.add(file); }} // dex Dex before the mergedoDexInject(context, loadedDex);
    }

    private static void doDexInject(Context appContext, HashSet<File> loadedDex) { String optimizeDir = appContext.getFilesDir().getAbsolutePath() + File.separator + OPTIMIZE_DEX_DIR; // data/data/ package name /files/optimize_dex (this must be the directory of your own program) File fopt = new File(optimizeDir);if(! fopt.exists()) { fopt.mkdirs(); } try {/ / 1. Load the application of dex PathClassLoader pathLoader = (PathClassLoader) appContext. GetClassLoader ();for(File dex : loadedDex) { // 2. Load the specified restored dex file DexClassLoader dexLoader = new DexClassLoader(dex.getabsolutePath (),// Directory where the restored dex (patch) resides Fopt.getabsolutepath (),// Decompression directory of dex (used for JAR, ZIP, and APK patches) null,// Library pathLoader required to load dex // parent class loader); // 3. Merge Object dexPathList = getPathList(dexLoader); Object pathPathList = getPathList(pathLoader); Object leftDexElements = getDexElements(dexPathList); Object rightDexElements = getDexElements(pathPathList); Object dexElements = combineArray(leftDexElements, rightDexElements); // Override Element in PathList [] dexElements; Assign Object pathList = getPathList(pathLoader); Do not use pathPathList, it will report an errorsetField(pathList, pathList.getClass(), "dexElements", dexElements); } } catch (Exception e) { e.printStackTrace(); }} /** * reflection reassigns attributes in an object */ private static voidsetField(Object obj, Class<? > cl, String field, Object value) throws NoSuchFieldException, IllegalAccessException { Field declaredField = cl.getDeclaredField(field); declaredField.setAccessible(true); declaredField.set(obj, value); Private static Object getField(Object obj, Class<? > cl, String field) throws NoSuchFieldException, IllegalAccessException { FieldlocalField = cl.getDeclaredField(field);
        localField.setAccessible(true);
        return localField.get(obj); } /** * Reflection gets the pathList Object in the classloader */ private static Object getPathList(Object baseDexClassLoader) throws ClassNotFoundException, NoSuchFieldException, IllegalAccessException {return getField(baseDexClassLoader, Class.forName("dalvik.system.BaseDexClassLoader"), "pathList"); Private static Object getDexElements(Object pathList) throws NoSuchFieldException, IllegalAccessException {return getField(pathList, pathList.getClass(), "dexElements"); } private static Object combineArray(Object arrayLhs, Object arrayRhs) {Class<? > componentType = arrayLhs.getClass().getComponentType(); int i = Array.getLength(arrayLhs); Int j = array.getLength (arrayRhs); int j = array.getLength (arrayRhs); Int k = I + j; Object result = Array.newinstance (componentType, k); // Create a new array of type componentType and length k. Arraycopy (arrayLhs, 0, result, 0, I); System.arraycopy(arrayRhs, 0, result, i, j);returnresult; }}Copy the code

The code is longer, but the comments are clear. Please look carefully. There are two things to be said here:

1) Class ref in pre-verified Class resolved to unexpected implementation

Object dexElements = combineArray(leftDexElements, rightDexElements); // Override Element in PathList [] dexElements; Assign Object pathList = getPathList(pathLoader); Do not use pathPathList, it will report an errorsetField(pathList, pathList.getClass(), "dexElements", dexElements);

Copy the code

After merging the Element array, be sure to retrieve the original pathList in the App again. Do not reuse the previous pathPathList. Class ref in pre-verified Class resolved to unexpected implementation

2) dexPath and optimizedDirectory directory problems

DexClassLoader dexLoader = new DexClassLoader(dex.getabsolutePath (),// Directory where dex (patch) is located fopt.getabsolutePath (),// Decompressed directory for storing dex (used for jar, ZIP, and APK patches) is null. // Library pathLoader required for loading dex // Parent class loaderCopy the code

The above code creates a DexClassLoader object, where the first and second parameters have one detail to note:

  • Parameter 1 is dexPath, which refers to all of the patch’s directories. It can be multiple directories (concatenated with colons), and it can be any directory, such as an SD card.

  • Parameter 2 is optimizedDirectory, which is the directory where the dex file extracted from the compressed package is stored, but it cannot be any directory. It must be the directory where the program belongs, for example: data/data/ package name/XXX.

If optimizedDirectory is specified as SD card directory, the following error will be reported:

java.lang.IllegalArgumentException: Optimized data directory /storage/emulated/0/opt_dex is not owned by the current user. Shared storage cannot protect your application from code injection attacks.

The SD card directory does not belong to the current user. OptimizedDirectory does not just store the dex file from the compressed package. If the patch file is a dex file, it copies the patch file to the optimizedDirectory.

3. Load the patch in JAR, APK, or ZIP format

It has been mentioned many times that DexClassLoader can load patch files in JAR, APK and ZIP formats. Is there any requirement for patch files in this format?

The answer is: the package must contain a dex file named classes.dex. According to? This is where you need to analyze the loadDexFile() method in the DexPathList class.

private static DexFile loadDexFile(File file, File optimizedDirectory) throws IOException {// If optimizedDirectory is null, it is the process of loading the dex File by PathClassLoaderif (optimizedDirectory == null) {
        returnnew DexFile(file); } // If optimizedDirectory is not null, this is how DexClassLoader loads the dex fileelse {
        String optimizedPath = optimizedPathFor(file, optimizedDirectory);
        returnDexFile.loadDex(file.getPath(), optimizedPath, 0); }}Copy the code

File, which may be a dex file, jar, APK, or ZIP file.

OptimizedPathFor () is the full path of the dex file in the optimizedDirectory after the dex file is loaded by the else branch:

private static String optimizedPathFor(File path, File optimizedDirectory) {
    String fileName = path.getName();
    if(! fileName.endsWith(DEX_SUFFIX)) { int lastDot = fileName.lastIndexOf("."); // If the patch does not have a suffix, add one to it".dex"The suffixif(lastDot < 0) { fileName += DEX_SUFFIX; } // No matter the patch suffix is dex, jar, apk, or zip, the final file in the optimizedDirectory must be dexelse {
            StringBuilder sb = new StringBuilder(lastDot + 4);
            sb.append(fileName, 0, lastDot);
            sb.append(DEX_SUFFIX);
            fileName = sb.toString();
        }
    }

    File result = new File(optimizedDirectory, fileName);
    return result.getPath();
}

Copy the code

As mentioned earlier, the Android classloader ultimately recognizes only dex files. Even if the patch is a jar, APk, zip, etc., it will extract the dex files, so the file name must end in dex. Ok, so the optimizedPathFor() method is not the point, but the else branch of loadDexFile() also has a dexfile.loaddex () method, which is quite important.

static public DexFile loadDex(String sourcePathName, String outputPathName, int flags) throws IOException {
    return new DexFile(sourcePathName, outputPathName, flags);
}

Copy the code

This method calls its own constructor and passes in the parameters. Then let’s look at the DexFile constructor:

/**
 * Open a DEX file, specifying the file in which the optimized DEX
 * data should be written.  If the optimized form exists and appears
 * to be current, it will be used; if not, the VM will attempt to
 * regenerate it.
 *
 * This is intended for use by applications that wish to download
 * and execute DEX files outside the usual application installation
 * mechanism.  This function should not be called directly by an
 * application; instead, use a class loader such as
 * dalvik.system.DexClassLoader.
 *
 * @param sourcePathName
 *  Jar or APK file with "classes.dex".  (May expand this to include
 *  "raw DEX" in the future.)
 * @param outputPathName
 *  File that will hold the optimized form of the DEX data.
 * @param flags
 *  Enable optional features.  (Currently none defined.)
 * @return
 *  A new or previously-opened DexFile.
 * @throws IOException
 *  If unable to open the source or output file.
 */
private DexFile(String sourceName, String outputName, int flags) throws IOException {
    if(outputName ! = null) { try { String parent = new File(outputName).getParent();if(Libcore.os.getuid() ! = Libcore.os.stat(parent).st_uid) { throw new IllegalArgumentException("Optimized data directory " + parent
                        + " is not owned by the current user. Shared storage cannot protect"
                        + " your application from code injection attacks.");
            }
        } catch (ErrnoException ignored) {
            // assume we'll fail with a more contextual error later } } mCookie = openDexFile(sourceName, outputName, flags); mFileName = sourceName; guard.open("close"); //System.out.println("DEX FILE cookie is " + mCookie + " sourceName=" + sourceName + " outputName=" + outputName); }Copy the code

Is it strange that I didn’t uncomment the constructor this time because it already has the desired answer in its comments:

@param sourcePathName Jar or APK file with "classes.dex".  (May expand this to include "raw DEX" in the future.)

Copy the code

This comment means that a jar or APK patch file needs to have a classes.dex. At this point, the requirements for the patch file in compressed format are clear. Then the next step is to generate patches in these formats and try them out. Making this kind of compressed file is also very simple, directly compressed into a ZIP file with compression software, and then change the suffix can be.

Six, test,

This part is actually not wanted to write, because it is relatively easy, but I think it is not complete, so let’s test a wave.

1, code,

1) the Activity

The layout file is just two buttons, it’s very simple and I don’t want to post the layout file code, just look at the click events of these two buttons.

public class SimpleHotFixActivity extends AppCompatActivity {

    @Override
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_simple_hot_fix); } public void fix(View View) {fixdexutils. loadFixedDex(this, Environment.getExternalStorageDirectory()); Public void clac(View View) {SimpleHotFixBugTesttest= new SimpleHotFixBugTest(); test.getBug(this); }}Copy the code

As you can see, the “fix” button click event is to load the patch file in the SD card directory.

2) SimpleHotFixBugTest

public class SimpleHotFixBugTest {
    public void getBug(Context context) {
        int i = 10;
        int a = 0;
        Toast.makeText(context, "Hello,I am CSDN_LQR:"+ i / a, Toast.LENGTH_SHORT).show(); }}Copy the code

What’s going to happen? The divisor is a zero exception, a simple runtime exception that can be fixed simply by changing the value of a to non-zero.

2, presentations,

1, bug

Not much to say, look at the operation.

ArithmeticException, no problem.

Caused by: java.lang.ArithmeticException: divide by zero

[Image upload failed…(image-bd95ED-1510735215247)]

2. Fix bugs dynamically

First, I put the patch file classes2.dex in the SD directory of my phone.

Then click the Repair button and then the Calculate button.

The zip patch is the same as the dex patch, just drop it in the SD card directory, but keep in mind that the zip patch is classes.dex!!

Finally, paste the Demo address

Github.com/GitLqr/HotF…

For more

Regular expressions will teach you in minutes

Technical Articles collection for the first half of 2017-184 articles grouped together

Advanced UI special like live like effect – a beautiful cool like animation

A small case of recording and playback

NDK project actual combat – high imitation 360 mobile phone assistant uninstall monitoring

Believe in yourself, there is nothing impossible, only unexpected

Wechat official account: Terminal R&D Department