Obfuscation is an essential skill in Android development on a daily basis. As long as we have experienced the process of App packaging online, more or less we need to understand some basic operations of code confusion. So what exactly is confusion? What are its benefits? What is the specific effect? Don’t worry, let’s explore its “unique” charm 🐳.

Confuse the introduction

Obfuscated code is the act of converting code in a program according to certain rules into code that is difficult to read and understand.

The benefits of confusion

The benefit of obfuscation is its purpose: to make APK difficult to reverse-engineer, that is, to substantially increase the cost of decompilation. In addition, obtrusiveness in Android can significantly reduce APK volume by removing unwanted resources at packaging time. Finally, there are workarounds to avoid the 64K method reference limit common in Android.

Let’s first look at the APK structure comparison before and after the confusion.

Confusion before:

After the confusion:

As can be seen from the above two figures, after confusion processing, package names, class names and member names in our APK were replaced with random and meaningless names, which increased the difficulty of code reading and understanding and increased the cost of decompilation. Careful observers may have noticed that the size of the APK decreased from 2.7M to 1.4M before and after the confusion, nearly doubling its size. Is it really that amazing? Ha ha, indeed is so magical, let us slowly uncover its mystery 😏.

Confusion in Android

In Android, what we call “obfuscation” actually has two meanings, one is the obfuscation of Java code and the other is the compression of resources. In fact, there is no correlation between these two, but it is habitually used together. So, with all that said, how do you turn obfuscation on on Android?

Enable the confusion

. android { buildTypes { release { minifyEnabledtrue
            shrinkResources true
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'}}}Copy the code

This is the basic operation to enable obfuscation by setting minifyEnabled to True. Also, set shrinkResources to true to enable resource compression. As you can see, obfuscation is usually enabled when releasing packages, and is not recommended in Debug mode because obfuscation adds extra compile time. Also, note that turning on resource compression only works if obfuscation is enabled! TXT in the above code represents the default obfuscation rule file provided by Android system, while proguard-rules.pro is the obfuscation rule we want to customize. As for how to customize the obfuscation rule, we will talk about 😄 in the following part.

Code confusion

In fact, the Java platform provides Proguard obfuscation tools to help you obfuscate code quickly. According to Java official introduction, the specific Chinese definition of Proguard is as follows:

  • It is a tool for code file compression, optimization, obfuscation, and validation
  • It can detect and remove useless classes, variables, methods, and properties
  • It optimizes bytecode and removes unused instructions
  • It confuses classes, variables, and methods by renaming them to nonsensical names
  • Finally, it verifies the processed code, mainly for Java 6 and above and Java ME

Resources compression

In Android, the compiler provides us with another powerful feature: resource compression. Resource compression helps us remove unused resources from project and dependency repositories, effectively reducing the size of APK packages. Resource compression and code obfuscation work together. Therefore, if resource compression needs to be enabled, it is important to enable code obfuscation first. Otherwise, the following problems may occur:

ERROR: Removing unused resources requires unused code shrinking to be turned on. See http://d.android.com/r/tools/shrink-resources.html for more information.
Affected Modules: app
Copy the code

Customize the resources to be reserved

When we enable resource compression, the system will remove all unused resources by default. If we need to preserve certain resources, we can create an XML file in our project marked with

(such as res/raw/keep.xml). Specify each resource to keep in the Tools: Keep attribute and each resource to discard in the Tools :discard attribute. Both attributes accept a comma-separated list of resource names. Similarly, we can use the character * as a wildcard. Such as:

<?xml version="1.0" encoding="utf-8"? >
<resources xmlns:tools="http://schemas.android.com/tools"
    tools:keep="@layout/activity_video*,@layout/dialog_update_v2"
    tools:discard="@layout/unused_layout,@drawable/unused_selector" />
Copy the code

Enable strict check mode

Under normal circumstances, the resource compressor can accurately determine whether the system is using resources. However, if your code (including the library) calls resources.getidentifier (), this means that your code will query for resource names based on dynamically generated strings. At this point, the resource compressor acts defensively by marking all resources with a matching name format as probably in use and cannot be removed. For example, the following code marks all resources prefixed with img_ as used:

String name = String.format("img_%1d", angle + 1);
res = getResources().getIdentifier(name, "drawable", getPackageName());
Copy the code

At this point, I can turn on the resource in strict review mode and only retain the resources that are sure to be used:

<?xml version="1.0" encoding="utf-8"? >
<resources xmlns:tools="http://schemas.android.com/tools"
    tools:shrinkMode="strict" />
Copy the code

Removing a Standby Resource

Gradle resource compressor only removes resources that are not referenced by the application, which means that it does not remove standby resources for different device configurations. If necessary, you can use the Android Gradle plugin’s resConfigs property to remove unwanted backup resource files (strings. XML for internationalization support, layout.xml for adaptation, etc.) :

android {
    defaultConfig {
        ...
        // Retain Chinese and English internationalization support
        resConfigs "en"."zh"}}Copy the code

Custom obfuscation rules

Having tasted the above “side dishes”, let’s savor the main course of this article: custom obfuscate rules. First, let’s look at common obfuscation commands.

Keep the command

The keep command refers to a series of commands that begin with -keep and are used primarily to preserve elements in Java that do not need to be confused. The following are common -keep commands:

  • -keep

    Function: Preserves the specified class and member to prevent confusion. Such as:

    Class and class members under com.moos.media. Entity
-keep public class com.moos.media.entity.**

# Save class NumberProgressBar-keep public class com.moos.media.widget.NumberProgressBar {*; }Copy the code

  • -keepclassmembers

    Function: Preserves the members (variables/methods) of the specified class, they will not be confused. Such as:

    Preserve class members: specific member methods in the MediaUtils class
-keepclassmembers class com.moos.media.MediaUtils {
    public static *** getLocalVideos(android.content.Context);
    public static *** getLocalPictures(android.content.Context);
}
Copy the code

  • -keepclasseswithmembers

    What it does: Preserves the specified class and its members (variables/methods), provided they are not deleted during the compression phase. Keep is used in a similar way to -keep:

    Class: BaseMediaEntity subclass-keepclasseswithmembers public class * extends com.moos.media.entity.BaseMediaEntity{*; }The OnProgressBarListener interface implementation class-keep public class * implements com.moos.media.widget.OnProgressBarListener {*; }Copy the code

  • @Keep

    In addition to the above, you can also choose to use the @keep annotation to preserve the desired code and prevent it from being confused. For example, we decorate a class with @keep to Keep it from being confused:

    @Keep
data class CloudMusicBean(var createDate: String,
                          var id: Long.var name: String,
                          var url: String,
                          val imgUrl: String)
Copy the code

    Similarly, we can have @keep decorate methods or fields to preserve them.

Other commands

  1. dontwarn

    The -dontwarn command is used when introducing a new library to handle warnings that cannot be resolved in the library. Such as:

    -keep class twitter4j.** { *; }

-dontwarn twitter4j.**
Copy the code

  2. For other commands, refer to the default obfuscation rules provided by the Android operating system:

    # do not generate mixed-case class names when obfuscating
-dontusemixedcaseclassnames

Do not skip classes for non-public libraries
-dontskipnonpubliclibraryclasses

# Log during obfuscation
-verbose

# Disable preverification
-dontpreverify

# Turn off optimizations
-dontoptimize

# Leave notes
-keepattributes *Annotation*

Keep all class names and local method names that have local methods
-keepclasseswithmembernames class * {
    native <methods>;
}

Keep the get and set methods for custom views
-keepclassmembers public class * extends android.view.View {
   void set* * * * (); *** get*(); }# preserve the Activity View and its subclass entry methods, such as onClick(Android.view.view)
-keepclassmembers class * extends android.app.Activity {
   public void *(android.view.View);
}

# keep enumeration
-keepclassmembers enum * {
    **[] $VALUES;
    public *;
}

Keep serialized classes
-keepclassmembers class * implements android.os.Parcelable {
  public static final android.os.Parcelable$Creator CREATOR;
}

Keep static members of R files-keepclassmembers class **.R$* { public static <fields>; } -dontwarn android.support.** -keep class android.support.annotation.Keep -keep @android.support.annotation.Keep class * {*; } -keepclasseswithmembers class * { @android.support.annotation.Keep <methods>; } -keepclasseswithmembers class * { @android.support.annotation.Keep <fields>; } -keepclasseswithmembers class * { @android.support.annotation.Keep <init>(...) ; }Copy the code

    More obfuscation commands can be found in Proguard’s most complete obfuscation rule description, which will not be explained here.

Confusing “blacklist”

After understanding the basic command of obfuscation, many of us are still at a loss as to what to obfuscate. In fact, ProGuard obfuscates most of the code in our project when we use code obfuscation. In order to prevent compile-time errors, we should use the keep command to keep some elements from being obfuscated. So, we just need to know which elements should not be confused:

The enumeration

Enumerations may inevitably be used in projects, but they should not be involved in the confusion. The reason: There is a values method inside the enumeration class, which is renamed and throws a NoSuchMethodException. Fortunately, the Android system’s default obliquity rules already add handling for enumerated classes, so we don’t need to do any extra work. To learn more about enumeration internal details can go to view the source code, space is limited no longer detailed.

The element being reflected

Classes, variables, methods, package names, etc. used by reflection should not be confused. The reason for this is that during code confusion, the element used by reflection is renamed, but reflection still finds the element by its previous name, so NoSuchMethodException and NoSuchFiledException often occur.

Entity class

Entity classes are often referred to as “data classes” and of course are often accompanied by serialization and deserialization operations. Many of us have also thought that confusion is to change the original “element” with a specific meaning into a meaningless name. Therefore, after the baptism of confusion, the key corresponding to the serialized value has become a meaningless field, which is definitely not what we want. At the same time, the deserialization process of creating an object is essentially a reflection process, and obfuscation can change the key, so it can also work against our expectations.

Four major components

The four components of Android should not be confused either. Here’s why:

  1. All four components need to be used beforeAndroidManifest.xmlFile registration declaration, however, after the obfuscating process, the four components of the class name will be tampered with, the actual use of the class andmanifestAn error occurred because the classes registered in.
  2. Other applications may use the package name of the class plus the class name when accessing the component. If the component is confused, the corresponding component may not be found or an exception may be generated.

Java method called by JNI

When a Java method called by JNI is obfuscated, the method name becomes a meaningless name that does not match the original Java method name in C++, and the called method cannot be found.

Others should not be confused

  • Custom controls need not be confused
  • JavaScript calling Java methods should not be confused
  • Java’s native methods should not be confused
  • Third-party libraries referenced in the project are also not recommended for confusion

Obfuscated stack trace

When code is obfuscated by ProGuard, it becomes difficult to read StackTrace information. Because method names and class names are obfuscated, it can be difficult to locate a crash problem even if it occurs. Fortunately, ProGuard provides a remedy. Before we proceed, let’s take a look at what happens with each build of ProGuard.

Obfuscate the output

Confuse building is completed, will be in the < module name > / build/outputs/mapping/release/directory to generate the following files:

  • dump.txt

    Describes the internal structure of all class files in APK.

  • mapping.txt

    Provides the content comparison table before and after the obfuscation, the content mainly contains the class, method and class member variables.

  • seeds.txt

    List the classes and members that are not obfuscated.

  • usage.txt

    List the code removed from APK.

Recovering the stack Trace

Now that we know what’s coming out of the obfuscated build, let’s look at the previous issue: StackTrace difficulty locating after obfuscating. How do YOU restore StackTrace positioning? The retrace tool, combined with the mapping.txt file mentioned above, can restore the scrambled crash StackTrace to the normal StackTrace. There are two main ways to restore StackTrace, so let’s use the following crash information as an example:

 java.lang.RuntimeException: Unable to start activity 
     Caused by: kotlin.KotlinNullPointerException
        at com.moos.media.ui.ImageSelectActivity.k(ImageSelectActivity.kt:71)
        at com.moos.media.ui.ImageSelectActivity.onCreate(ImageSelectActivity.kt:58)
        at android.app.Activity.performCreate(Activity.java:6237)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1107)
Copy the code

  1. Through the Retrace script tool

    First, go to the /tools/ proGuard /bin directory of the Android SDK path. Here, take the Mac system as an example, you can see the following contents:

    You can see the above three files, and proGuardgui.sh is the retrace script we need (proGuardgui.bat on Windows). On Windows, just double-click proGuardgui.bat to run it. On Mac, if you haven’t done any configuration, drag proGuardgui.sh to the Mac terminal and press Enter to run it. We should then see the following interface:

    Select the ReTrace bar and add the location of the mapping. TXT file generated from our Obfuscated stack Trace file. Then copy our Obfuscated crash information to the Obfuscated Stack Trace column and click ReTrace! Button to restore our crash log information, the result is shown above, our previous confusion log: At com. Moos. Media. UI. ImageSelectActivity. K (ImageSelectActivity. Kt: 71) reduction into the at Com. Moos. Media. UI. ImageSelectActivity. InitView (ImageSelectActivity. Kt: 71). ImageSelectActivity. K is the method name, after we confuse ImageSelectActivity. InitView is originally not confuse the method name, before resorting to ReTrace tool’s help, we can as soon as before positioning the collapse area code.

  2. Through the retrace command line

    TXT file (for example, proguard_stacktrace.txt), and then run the following command:

    retrace.sh -verbose mapping.txt proguard_stacktrace.txt
Copy the code

    If you are running Windows, run the following command:

    retrace.bat -verbose mapping.txt proguard_stacktrace.txt
Copy the code

    The final restore result is the same as before:

    You may find an Unknown Source problem when restoring stackTrace in either of the following ways:

Note that we need to add the following configuration to the obfuscation rule to improve our StackSource lookup efficiency:

Keep the source file name and line number
-keepattributes SourceFile,LineNumberTable
Copy the code

In addition, every time we created a release build using ProGuard we overwrote the mapping.txt file from the previous release, so we had to be careful to save a copy every time we released a new release. By keeping a copy of the mapping.txt file for each release build, we can debug and fix problems with older applications on user-submitted scrambled StackTrace.

Operation of rising posture

After the introduction above, we know that after code obfuscation, APK package name, class name and member name are transformed into meaningless and difficult to understand names, increasing the cost of decompilation. Android ProGuard provides us with a default “obtrusion dictionary”, which converts element names to lowercase letters. So, can we define our own obfuscation dictionary? Just to keep you in suspense, let’s take a look at a rendering:

Isn’t this wave manipulation a bit “outstanding”? Proguard-rules.pro obfuscation rule proguard-rules.pro obfuscation rule proguard-rules.pro obfuscation rule proguard-rules.pro

The obliquity dictionary used in this article can be viewed and downloaded here: proguard_tradition.txt

Of course, you can customize your obfuscation dictionary to make it more difficult to decompile.

Along the way, we found that obfuscation technology is worth further study and research in terms of its necessity and advantages. This article only gives you a taste of the tip of the iceberg. Due to my limited technical level, if you find any problems or improper explanation, please kindly point out and correct.

Relevant reference

  • Shrink your app

  • Read code obfuscation in Android

  • Practical ProGuard rules example

  • The Android ProGuard code confuses those things

  • Proguard most complete obfuscation rule description