The principle of analysis

DexClassLoader mechanism

Process implementation

Let’s start with a piece of code for display. Full version of the code in hot Fix cold startup module:hotfix_dexload

Unrepaired function

// Class to be repaired
package com.a.fix;
public class M {
    public static String a(a){return  "M aaa";}
}

// The class used to display data
package com.a.android_sample;
public class MainActivity extends AppCompatActivity {
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); String str = M.a(); ((TextView) findViewById(R.id.tv)).setText(str)); }}Copy the code

Class repair code

package com.a.fix;
public class M {
    public static String a(a){return  "M aaa fix";}
}
Copy the code

Store patch. Dex

Copy patch.dex to the assets folder

Where to put patch.dex as long as it’s readable when the app starts. Of course you can put it on an SD card. We select assets, and when the program starts, we copy it to where we want it to go.

validation

At this point, our repair function is implemented. I tried Android7 and everything is OK.

The phenomenon of

The above code, we try to run on Android4.4 and below, the result is wrong.

java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation
at com.a.android_sample.MainActivity.onCreate(MainActivity.java16) :at android.app.Activity.perfromCreate(Activity.java: 5266).at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java: 1313).at android.app.ActivityThread.performLaunchActivity(ActivityThread.java: 3733).at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java: 3939).Copy the code

Error code

String str = M.a();
Copy the code

In simple terms

1. If class A and its reference classes are in the same dex, class A is verified and optimized in advance and marked CLASS_ISPREVERIFIED. Here, MainActivity is marked. 2. When we call M.a(), we need to load class M, and the virtual machine checks whether M and MainActivity belong to the same dex. It’s clearly not there. That’s an error.

Do not understand, Dalvik class loading mechanism, this reason is not analyzed. We’re standing on the shoulders of giants, not ponies crossing the river.

Call link

This part can be folded without looking.

M.a()

Install the plugin java2smali for AndroidStudio and see what MainActivity builds. Mainactivity. smali part of the code

.class public Lcom/a/android_sample/MainActivity;
.source "MainActivity.java"

.method protected onCreate(Landroid/os/Bundle;)Registers 4 # Error executing to this line. .line 16 invoke-static {}, Lcom/a/fix/M; ->a()Ljava/lang/String; .line17
    ...
    invoke-virtual {v1, v0}, Landroid/widget/TextView;->setText(Ljava/lang/CharSequence;)V
    ...
.end method
Copy the code

dvmResolveMethod

Dalvik /vm/oo/ resolve. CPP

/* * Find the method corresponding to "methodRef". * If this is a static method, we ensure that the method's class is * initialized. */
// Some code is omitted
Method* dvmResolveMethod(const ClassObject* referrer, u4 methodIdx,
    MethodType methodType){
    ClassObject* resClass;
    const DexMethodId* pMethodId;
    pMethodId = dexGetMethodId(pDvmDex->pDexFile, methodIdx);

    // This is where we start calling the specific code throws we mentioned in the previous section.
    resClass = dvmResolveClass(referrer, pMethodId->classIdx, false);
    if (resClass == NULL) {
        /* can't find the class that the method is a part of */
        assert(dvmCheckException(dvmThreadSelf()));
        returnNULL; }... }Copy the code

reference

An in-depth understanding of the Java Virtual Machine: Advanced features and Best Practices for the JVM (version 3). PDF An in-depth understanding of the Dalvik Virtual Machine System source code (AOSP) github address link to download what you want. Or this official website links to android App hot patch dynamic repair technology to introduce android hot patch pre-verify problems and practice 05-DALVIK load and parse DEX process

Project analysis

We are copying the code, and we will only get an error if all three conditions are met

// Some code is omitted
ClassObject* dvmResolveClass(const ClassObject* referrer, u4 classIdx,
    bool fromUnverifiedConstant){
    
    resClass = dvmDexGetResolvedClass(pDvmDex, classIdx);
    if(resClass ! = NULL)return resClass;
    
    if(! fromUnverifiedConstant && IS_CLASS_FLAG_SET(referrer, CLASS_ISPREVERIFIED)) { ClassObject* resClassCheck = resClass;if(referrer->pDvmDex ! = resClassCheck->pDvmDex && resClassCheck->classLoader ! = NULL){ dvmThrowIllegalAccessError("Class ref in pre-verified class resolved to unexpected "
                    "implementation");
                returnNULL; }}}return resClass;
}
Copy the code

Based on the above code, there are roughly four solutions.

• Do not mark the dexopt process with CLASS_ISPREVERIFIED

Q-zone piling scheme breaks this limitation, but results in preVerify failure and loss of performance.

• Modify fromUnverfiedConstant = true

It is necessary to intercept the native hook system method, change the entry parameters of the method, and change fromUnverifiedConstant to true uniformly, which is risky and almost not adopted by anyone. Cydia native hook

• DvmDexGetResolvedClass does not return null

QFix uses this solution,

• The patch class is in the same dex as the reference class

Tinker and other total synthesis schemes break through this limitation.

Q-zone pile insertion scheme

Project analysis

Using bytecode technology, a reference to Hackcode. class is inserted into the constructor of each class, causing MainActivity to refer to hack. class in hack.dex, causing Verify to fail. At this point the scheme is divided into two parts

• Package hackcode.class separately
• MainActivity references hackcode.class.

package com.a.hack;
public class HackCode {}
Copy the code

Where the actual code is executed.

//dalvik/vm/analysis/CodeVerify.cpp
case OP_CONST_CLASS:
	 // If it fails, the error value is set to failure.
      resClass = dvmOptResolveClass(meth->clazz, decInsn.vB, &failure);

////dalvik/vm/analysis/Optimize.cpp
/* * Performs access checks on every resolve, Mother and to acknowledge the existence of classes * defined in more than one DEX file. * Classes defined in multiple DEX are not recognized */
ClassObject* dvmOptResolveClass(ClassObject* referrer, u4 classIdx, VerifyError* pFailure){...const char* className = dexStringByTypeIdx(pDvmDex->pDexFile, classIdx);
    // Referrer is a hack.class for all reference classes including MainAcitivityClass and resClass
    Hack.class is not found in the referrer's dex
	resClass = dvmFindClassNoInit(className, referrer->classLoader);
    if (resClass == NULL) { *pFailure = VERIFY_ERROR_NO_CLASS; . }... }Copy the code

Reference hackCode. Class

Apk source code cannot include hackcode.class, we insert references via bytecode. Write a custom Gradle plug-in, using Javassist bytecode technology custom Gradle plug-in referenceGradle – Groovy, Gradle, and custom Gradle pluginsJavassist referenceJavassist uses full parsing Key code, a little long

class HackTransform extends Transform {

    def pool = ClassPool.default
    def project
    ....	
    @Override
    void transform(TransformInvocation transformInvocation) throws javax.xml.crypto.dsig.TransformException, InterruptedException, IOException {
        super.transform(transformInvocation)

        project.android.bootClasspath.each {
            pool.appendClassPath(it.absolutePath)
        }
        // This line should be careful, otherwise the compilation will not pass
        pool.makeClass("com.a.hack.HackCode")

        transformInvocation.inputs.each {

            it.jarInputs.each {
                pool.insertClassPath(it.file.absolutePath)
                // Rename output file (conflict with directory copyFile)
                def jarName = it.name
                def md5Name = DigestUtils.md5Hex(it.file.getAbsolutePath())
                if (jarName.endsWith(".jar")) {
                    jarName = jarName.substring(0, jarName.length() - 4)}def dest = transformInvocation.outputProvider.getContentLocation(
                        jarName + md5Name, it.contentTypes, it.scopes, Format.JAR)
                org.apache.commons.io.FileUtils.copyFile(it.file, dest)
            }

            it.directoryInputs.each {
                def inputDir = it.file.absolutePath
                pool.insertClassPath(inputDir)
                findTarget(it.file, inputDir)
                def dest = transformInvocation.outputProvider.getContentLocation(
                        it.name, it.contentTypes, it.scopes, Format.DIRECTORY)
                org.apache.commons.io.FileUtils.copyDirectory(it.file, dest)
            }
        }
    }

    private void findTarget(File fileOrDir, String inputDir) {
        if (fileOrDir.isDirectory()) {
            fileOrDir.listFiles().each {
                findTarget(it, inputDir)
            }
        } else {
            modify(fileOrDir, inputDir)

        }
    }

    private void modify(File file, String fileName) {
        def filePath = file.absolutePath

        if(! filePath.endsWith(SdkConstants.DOT_CLASS) ||filePath.contains('R$') 
           || filePath.contains('R.class')
           || filePath.contains("BuildConfig.class")) {
            return
        }
        def className = filePath.replace(fileName, "")
        		.replace("\ \".".").replace("/".".")
        def name = className.replace(SdkConstants.DOT_CLASS, "").substring(1)
        CtClass ctClass = pool.get(name)
        // Our custom Application is the initial class, and the Hakcode reference can only be inserted after loading the dex class.
        if(ctClass.getSuperclass() ! =null
                && ctClass.getSuperclass().name == "android.app.Application") {
            return
        }
       
        // Where the insertion of the bytecode is actually performed
        ctClass.defrost()
        CtConstructor[] constructors = ctClass.getDeclaredConstructors()
        if(constructors ! =null && constructors.length > 0) {
            CtConstructor constructor = constructors[0]
            def body = "android.util.Log.e(\"alvin\",\"${constructor.name} constructor\" + com.a.hack.HackCode.class);"
            constructor.insertBefore(body)
        }
        ctClass.writeFile(fileName)
        ctClass.detach()
    }
}
Copy the code

Generate hack. Dex

Refer to the generation method of patch.dex. Write the app/main/Java/com/a/hack/HackCode. Java, separate into dex, after generated, Java can delete this file.

package com.a.hack;
public class HackCode {}
Copy the code

// Go to the Java source directory,
cd app/main/java
/ /. Class files
javac com/a/hack/HackCode.java   
/ / generated hack. Dex
dx --dex --output com/a/hack/hack.dex com/a/hack/HackCode.class
Copy the code

Load the hack. Dex

Refer to patch.dex.

validation

Verified successfully on android4.4

Cydia NativeHook

You need to intercept the native hook system method by changing the entry parameter of the method to true.

Here, we adopt Cydia Substrate, hook dvmResolveClass method, the steps are as follows: hook implementation and dynamic library download, note that the scheme is only feasible on Android4.4.

Implementation steps

Cydia so library and header files

You can download it here. Put the so library in its own directory for example

<moduleName>/src/main/jniLibs/armeabi-v7a/libsubstrate.so
<moduleName>/src/main/jniLibs/armeabi-v7a/libsubstrate-dvm.so
Copy the code

Import header file

<moduleName>/src/main/cpp/include/substrate.h
Copy the code

Hook code implementation

//<moduleName>/src/main/cpp/cydia-hook.cpp
#include "include/substrate.h"
#include <android/log.h>

#define TAG "alvin"
#define LOGE(...). __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)

// Old function pointer to old function
void *(*oldDvmResolveClass)(void *referrer, unsigned int classIdx, bool fromUnverifiedConstant);

// New function implementation
void *newDvmResolveClass(void *referrer, unsigned int classIdx, bool fromUnverifiedConstant) {
    // Here, fromUnverifiedConstant is mandatory to true, so you don't check whether the dex is equal.
    return oldDvmResolveClass(referrer, classIdx, true);
}

// Specify the lib to hook, involving the dvmResolveClass so
MSConfig(MSFilterLibrary, "/system/lib/libdvm.so")
// Specify the application to hook
MSConfig(MSFilterExecutable, "com.a.dexload.cydia")

MSInitialize {
    MSImageRef image = MSGetImageByName("/system/lib/libdvm.so");
    if (image == NULL) {
        return;
    }
    void *resloveMethd = MSFindSymbol(image, "dvmResolveClass");
    if (resloveMethd == NULL) {
        return;
    }
    // Concrete Hook implementation
    MSHookFunction(resloveMethd, (void *) newDvmResolveClass, (void **) &oldDvmResolveClass);
}
Copy the code

CMakeLists.txt

Generate libcydiahook. So

cmake_minimum_required(VERSION 3.102.)

add_library(cydiahook SHARED src/main/cpp/cydia-hook.cpp)
target_include_directories(cydiahook PRIVATE  ${CMAKE_SOURCE_DIR}/src/main/cpp/include)
find_library(log-lib log)
file(GLOB libs ${CMAKE_SOURCE_DIR}/src/main/jniLibs/armeabi-v7a/libsubstrate.so ${CMAKE_SOURCE_DIR}/src/main/jniLibs/armeabi-v7a/libsubstrate-dvm.so)
target_link_libraries( cydiahook   ${libs}  ${log-lib})
Copy the code

Libcydiahook. So loading

public class ApplicationApp extends Application {
    static {
        System.loadLibrary("cydiahook"); }}Copy the code

other

ClassObject properties

As with Andfix, we can introduce a dexfile.h header file that can turn arguments and results into actual class objects and view some of the class properties

 // New function implementation
void *newDvmResolveClass(void *referrer, unsigned int classIdx, bool fromUnverifiedConstant) {
    
    void *res = oldDvmResolveClass(referrer, classIdx, true);

    ClassObject *referrerClass = reinterpret_cast<ClassObject *>(referrer);
	ClassObject *resClass = reinterpret_cast<ClassObject *>(res);
	if (resClass == NULL) {
        LOGE("newDvmResolveClass %s, %s", referrerClass->descriptor,
             "resClass is NULL");
    } else {
        LOGE("newDvmResolveClass %s, %s", referrerClass->descriptor,
             resClass->descriptor);
    }
    return res;
}
Copy the code

risk

Similar to Andfix,native Hook has various compatibility and stability problems, and even security problems. At the same time, it intercepts a method that involves dalvik’s basic functions and is called frequently, which will undoubtedly be much riskier.

QFix scheme implementation

The principle can be found in this article: QFix Road to Discovery – Manual Q hot patch lightweight solution

plan

Go back to this figure and start with dvmResolveClass method to resolve the Patch class in advance.The initial solution was to create a class with the “const-class” or “instance-of” directives, fromUnverifiedConstant = true, to bypass the dex detection. And it worked. But there are two problems:

• How do I know which patch classes in advance?
• Or simply refer to all classes? Performance issues? How to do that?

public class ApplicationApp extends Application {
    @Override
    protected void attachBaseContext(Context base) {
        super.attachBaseContext(base);

        DexInstaller.installDex(base, this.getExternalCacheDir().getAbsolutePath() + "/patch.dex");
		// The const-class directive is executed
        Log.d("alvin"."bug class:" + com.a.fix.M.class);
}    
Copy the code

QFix gives up the scheme of directly loading patch class. After analysis,

• The number of classes in a patch pack is limited.
• The number of DEX files in APK is also limited.

The following scheme is obtained:

• When apK is constructed, dex is pre-embedded with blank classes, and the associated files of each dex and blank classes are obtained.
• Build the patch package and map the association between the blank class of bug dex and the patch class in the original dex classIdx.
• ———– Run the app and load the patch package ————-
• Using the Java method, call classLoader.loadClass(blank class name)
• Using the JNI method, call dvmFindLoadedClass(blank class Descriptor)
• Using jni method, called dvmResolveClass (referrer: blank, classIdx, fromUnverifiedConstant: true)

As for how to find this method, of course, is the source inside the wandering.

In field

All the implementation code is in Github

Blank classes are injected into Dex

Custom Gradle plugin, use smali to manipulate dexfile and inject class.

• Custom Gradle plugin reference
• Smali technical reference
• Class is injected into dex

BuildSrc /build.gradle adds dependencies

//buildSrc/build.gradle
dependencies {
 	...
    compile group: 'org.smali'.name: 'dexlib2'.version: '2.2.4'. }Copy the code

2. The plugin code

class QFixPlugin implements Plugin<Project> {

    void apply(Project project1) {
        project1.afterEvaluate { project ->
            project.tasks.mergeDexDebug {
                doLast {
                    println 'QFixPlugin inject Class after mergeDexDebug'
                    project.tasks.mergeDexDebug.getOutputs().getFiles().each { dir ->
                        println "outputs: " + dir
                        if(dir ! =null && dir.exists()) {
                            def files = dir.listFiles()
                            files.each { file ->
                                String dexfilepath = file.getAbsolutePath()
                                println "Outputs Dex file's path: " + dexfilepath
                                  InjectClassHelper.injectHackClass(dexfilepath)
                            }
                        }
                    }
                }
            }
        }
    }
}

Copy the code

InjectClassHelper.java

public class InjectClassHelper {

    public static void injectHackClass(String dexPath) {
        try {
            File file = new File(dexPath);
            String fileName = file.getName();
            String indexStr = fileName.split("\ \.") [0].replace("classes"."");
            System.out.println(" =============indexStr:"+indexStr);
            String className = "com.a.Hack"+ indexStr;
            String classType = "Lcom/a/Hack" + indexStr + ";";
            
            DexBackedDexFile dexFile = DexFileFactory.loadDexFile(dexPath, Opcodes.getDefault());
			ImmutableDexFile immutableDexFile = ImmutableDexFile.of(dexFile);

            Set<ClassDef> classDefs = new HashSet<>();
            for (ImmutableClassDef classDef : immutableDexFile.getClasses()) {
                classDefs.add(classDef);
            }
            ImmutableClassDef immutableClassDef = new ImmutableClassDef(
                    classType,
                    AccessFlags.PUBLIC.getValue(),
                    "Ljava/lang/Object;".null.null.null.null.null);
            classDefs.add(immutableClassDef);

            String resultPath = dexPath;
            File resultFile = new File(resultPath);
            if(resultFile ! =null && resultFile.exists()) resultFile.delete();
            DexFileFactory.writeDexFile(resultPath, new DexFile() {
                
                @Override
                public Set<ClassDef> getClasses(a) {
                    return new HashSet<>(classDefs);
                }

                @Override
                public Opcodes getOpcodes(a) {
                    returndexFile.getOpcodes(); }}); System.out.println("Outputs injectHackClass: " + file.getName() + ":" + className);
            
        } catch(Exception e) { e.printStackTrace(); }}}Copy the code

Mapping

Outputs Dex file's path: /Users/mawenqiang/Documents/demo_project/hotfix_dexload/dexload_QFix/build/intermediates/dex/debug/out/classes2.dex
Outputs injectHackClass: classes2.dex:com.a.Hack2
Outputs Dex file's path: /Users/mawenqiang/Documents/demo_project/hotfix_dexload/dexload_QFix/build/intermediates/dex/debug/out/classes.dex
Outputs injectHackClass: classes.dex:com.a.Hack
Copy the code

Run the dexdump command

#dexdump -h classes2.dex > classes2.dump

Class #1697 header:
class_idx           : 2277 #class_idx
......
Class descriptor  : 'Lcom/a/fix/M; '.Copy the code

We can get mapping.txt

classes2.dex:com.a.Hack2:com.a.fix.M:2277
Copy the code

Import patch.dex and Mapping.text

load patch.dex

The generation and loading of patch.dex remain unchanged, as shown above.

Resolve patch M.c lass

Also in ApplicationApp. AttachBaseContext (), in the load patch after execution. The code file applicationApp.java

• Analytical Mapping. TXT, get hackClassName patchClassIdx
• classLoader.loadClass(com.a.Hack2)
• nativeResolveClass(hackClassDescriptor, patchClassIdx)

public static void resolvePatchClasses(Context context) {
        try {
            BufferedReader br = new BufferedReader(new FileReader(context.getExternalCacheDir().getAbsolutePath() + "/classIdx.txt"));
            String line = "";
            while(! TextUtils.isEmpty(line = br.readLine())) { String[] ss = line.split(":");
                //classes2.dex:com.a.Hack2:com.a.fix.M:2277
                if(ss ! =null && ss.length == 4) {
                    String hackClassName = ss[1];
                    long patchClassIdx = Long.parseLong(ss[3]);
                    Log.d("alvin"."readLine:" + line);
                    String hackClassDescriptor = "L" + hackClassName.replace('. '.'/') + ";";
                    Log.d("alvin"."classNameToDescriptor: " + hackClassName + "-- >" + hackClassDescriptor);
                    ResolveTool.loadClass(context, hackClassName);
                    ResolveTool.nativeResolveClass(hackClassDescriptor, patchClassIdx);
                }
            }
            br.close();

        } catch(Exception e) { e.printStackTrace(); }}/**
     * * "descriptor" should have the form "Ljava/lang/Class;" or
     * * "[Ljava/lang/Class;", i.e. a descriptor and not an internal-form
     * * class name.
     *
     * @param referrerDescriptor
     * @param classIdx
     * @return* /
    public static native boolean nativeResolveClass(String referrerDescriptor, long classIdx);

    public static void loadClass(Context context, String className) {
        try {
            Log.d("alvin", context.getClassLoader().loadClass(className).getSimpleName());
        } catch (Exception e) {
            e.printStackTrace();
            Log.d("alvin", e.getMessage()); }}Copy the code

NativeResolveClass is a normal JNI method, and the code is actually simple.

#include <jni.h>
#include <android/log.h>
#include <dlfcn.h>

#define  LOG_TAG    "alvin"
#define  LOGE(...)  __android_log_print(ANDROID_LOG_ERROR,LOG_TAG,__VA_ARGS__)

// Method pointer
void *(*dvmFindLoadedClass)(const char *);

// Method pointer
void *(*dvmResolveClass)(const void *, unsigned int.bool);


extern "C" jboolean Java_com_a_dexload_qfix_ResolveTool_nativeResolveClass(JNIEnv *env, jclass thiz, jstring referrerDescriptor, jlong classIdx) {
    LOGE("enter nativeResolveClass");
    void *handle = 0;
    handle = dlopen("/system/lib/libdvm.so", RTLD_LAZY);
    if(! handle)LOGE("dlopen libdvm.so fail");
    if(! handle)return false;

    const char *loadClassSymbols[3] = {
            "_Z18dvmFindLoadedClassPKc"."_Z18kvmFindLoadedClassPKc"."dvmFindLoadedClass"};
    for (int i = 0; i < 3; i++) {
        dvmFindLoadedClass = reinterpret_cast<void* (*) (const char> (*)dlsym(handle, loadClassSymbols[i]));
        if (dvmFindLoadedClass) {
            LOGE("dlsym dvmFindLoadedClass success %s", loadClassSymbols[i]);
            break; }}const char *resolveClassSymbols[2] = {"dvmResolveClass"."vResolveClass"};
    for (int i = 0; i < 2; i++) {
        dvmResolveClass = reinterpret_cast<void* (*) (const void *, unsigned int.bool) > (dlsym(handle, resolveClassSymbols[i]));
        if (dvmResolveClass) {
            LOGE("dlsym dvmResolveClass success %s", resolveClassSymbols[i]);
            break; }}if(! dvmFindLoadedClass)LOGE("dlsym dvmFindLoadedClass fail");
    if(! dvmResolveClass)LOGE("dlsym dvmResolveClass fail");
    if(! dvmFindLoadedClass || ! dvmResolveClass)return false;

    const char *descriptorChars = (*env).GetStringUTFChars(referrerDescriptor, 0);
    / / referrerClassObj is com. A.H ack2
    void *referrerClassObj = dvmFindLoadedClass(descriptorChars);
    dvmResolveClass(referrerClassObj, classIdx, true);
    return true;
}
Copy the code

At this point, the code is fully implemented.