When we advertised Shadow, we said it had two major features, one of which was called the “full dynamic plug-in framework.” This article is about that feature. We’ve long used a plugin framework based on hundreds of proprietary reflection apis. As mentioned in previous articles, this plugin framework is constantly compatible with new versions of Android and OEM systems. The Attach method of the Activity, in particular, often requires compatibility. Overriding methods are occasionally added to the shellactivity as well. But in practice, none of these needs can be met. This is because we originally packaged the plug-in framework in the host, just like the other plug-in frameworks you see on the market. Therefore, these fixes and updates only take effect in the next version of the host. Bug fixes are fine, and the next version is fine, but new features are a problem, and you need to make sure that plug-ins that use new features are not launched by the hosts of older versions.

The fundamentals of dynamics

Let’s go straight back to the fundamentals of dynamics and show how Shadow applies them. The basic principle of dynamics is very simple and basic to Java. But there are a lot of people who don’t understand it thoroughly and don’t use it flexibly.

Java code is compiled without linking procedures. The linking process refers to the traditional C compilation process that is divided into two steps. One is to compile the source code into machine code, and temporarily replace symbols that reference other files (such as global variables in files) with symbolic names. Then the second step is the linking step, where the symbolic name temporarily used in the previous step is actually replaced with the actual memory address. From a Java perspective, this process is equivalent to Java source code being compiled into bytecode, and other classes referenced in the source code being temporarily replaced with names in bytecode. You then replace the name in the bytecode with the actual implementation code of the other class during a linking process. The reality, however, is that Java does not have this linking process. Java compiles bytecodes that hold the names of other classes. Implementations of other classes are found at run time. Therefore, this process is equivalent to the dynamic linking process of C language. The C compilation process we mentioned earlier is called static chaining. So, some people who have studied C have commented on Java and said, Java is a completely dynamically linked language, it’s a dynamic language. By “dynamic language” I mean that the linking process is dynamic. Java is a statically typed language. Don’t confuse the two.

With the exception of some special system classes optimized for Native implementations, Java classes are dynamically loaded by the ClassLoader at runtime. If class A refers to class B, the implementation of class B will be looked up to the ClassLoader that loaded it when the code of class A executes to use B. If you find an implementation of class B, you won’t be able to create an instance of class B, continue execution, or call a static method of class B. And classes with the same name loaded by the same ClassLoader are actually the same class at runtime. Public static final int A public static final int A public static final int A It prints out different values, right? The answer should be “maybe”. Because under A carefully constructed ClassLoader structure, classes B and C may be loaded by different classloaders, the implementations of class A they request from their respective Classloaders may be different. Even though there is only one implementation of class A, class B and class C load into two different classes of class A. Once you modify the A-static domain of one with reflection, the A-static domain of the other does not change.

Java also has two features related to dynamics, one is interfaces and the other is upward transformation.

Class<? > implClass = classLoader.loadClass("com.xxx.AImpl");
Object implObject = implClass.newInstance();
A a = (A) implObject;
Copy the code

This assumes that the classLoader dynamically loads some Java classes, one of which is called com.xxx.AImpl, and AImpl either inherits from A or implements the A interface. Note that casting is used here because the code level is casting Object down to A. But we actually know that the implObject type is AImpl, and the conversion of AImpl to A is an upward transformation. Upward transition is always safe. So it’s always possible to define the interface first, and design it so that it’s generic and stable. As long as the interface does not change, its implementation can always be modified. We package the interface in the host, so the interface can’t be updated easily. But its implementation can always be updated.

In all plug-in frameworks, the Activity is loaded like this: new a DexClassLoader loads the plug-in APK. Then load the specified Activity name from the plug-in ClassLoader, and then convert it to the Activity type after newInstance. In fact, the Android system itself does this when it starts an Activity. So this is the rationale behind the plug-in mechanism for dynamically updating activities.

Therefore, the problem all plug-in frameworks solve is not how to dynamically load classes, but how to dynamically load activities that are not registered in the AndroidManifest, how to run properly. If Android didn’t have the limitations of AndroidManifest, then there would be no need for any plug-in framework. Because the Java language itself supports the ability to dynamically update implementations.

Dynamic Manager

The function of Shadow Manager is to manage plug-ins, including plug-in download logic, entry logic, preloading logic, etc. It’s all before the Loader.

Since Manager is a normal class, not one that the Android system requires to be registered in the Manifest, dynamic Manager is a generic dynamic loading implementation.

To keep the fixed code in the host as small as possible, the interface we define for the Manager is an interface similar to the traditional Main function.

void enter(Context context, long formId, Bundle bundle, EnterCallback callback);
Copy the code

This is the only method of the Manager, and the only method that is called in the host. Pass the current interface Context to open the next plug-in Activity. Pass all possible parameters from the plug-in to the plug-in through the Bundle. Define some fromId so that the Manager’s implementation logic can tell where the Enter is coming from this time. In fact, each enter call in the host can set a different fromId, which lets the Manager know which line of code the call came from in the host. Pass in an EnterCallback so that the Manager can return a dynamically loaded View as the Loading View of the plug-in.

Dynamic of Loader

Loader is responsible for loading the plug-in Activity, and then implement the plug-in Activity lifecycle and other functions of the core logic. Many plug-in frameworks have only Loader functionality, or open source Loader functionality. In general, a Loader is a bridge between the host and the plug-in. For example, we need to execute Loader code in the host to Hack some system classes to load the plug-in Activity. Or in the surrogate shell Activity of the host, use the Loader to load the plug-in Activity to complete the callback function. So usually the host code depends directly on the Loader code. This is why other plug-in frameworks need to package the plug-in framework’s own code in the host.

A slightly more complicated issue is that the proxy shell ContainerActivity and PluginActivity need to call each other via the Loader. So when Shadow applies the dynamic principle mentioned earlier, it has a bidirectional interface and you can see HostActivityDelegate and HostActivityDelegator in the code. Defining these interfaces prevents ContainerActivity and Loader from loading other classes that each other depends on. Defined as an interface, you just need to load the interface.

With this design, most of the code that needs to be modified or fixed in the plug-in framework can be published dynamically. In addition, there can be multiple Loaders with different implementations in the same host. In this way, services can modify Loader codes based on their own bugs without affecting other services. Loader can also couple business logic in emergency situations.

The dynamic of Container

Containers are proxy shells registered in the host AndroidManifest. Since activities are created by the system based on the name of the Activity directly through the host’s PathClassLoader, these activities must be packaged in the host to be in the PathClassLoader and found by the system. Therefore, the Container cannot be loaded into the Loader through the general dynamic loading method. Because the general method mentioned above is to load a new ClassLoader dynamically.

But the host of our business is very strict about the deltas of the merged code, requiring zero deltas. That is, as we merge in the code, we have to optimize the original code so that the whole thing increases by 0. The increment includes the increment in the volume of the installation package and the increment in the number of methods.

Therefore, it is not enough to make the Loader dynamic, because many methods need to be Override in advance on the proxy shell Activity. Since the Delegate and Delegator interfaces are defined, and the superOnCreate and other methods are added to the Delegator interface, four more methods are added to the Activity for every method that requires Override. There are about 350 methods on the Activity.

The implementation of the Container is very simple due to the previous Loader dynamics. Whatever method is invoked to the Delegate interface, it does not implement any logic on its own. It is reasonable to assume that there are no bugs, at least if the methods are fully implemented, the Container can be used for a long time even if it is not dynamic.

Android virtual machines differ from regular JVMS in that they can modify the private final domain through reflection. This does not work on a normal JVM, and those of you who have read Ideas for Java programming may remember this section. The parent domain of the ClassLoader class happens to be the private final domain. The parent of a ClassLoader refers to the parent of a ClassLoader, that is, the parent of a ClassLoader. The host PathClassLoader is a ClassLoader with normal “parent delegate” logic that delegates its parent to load any class before it loads it. If the parent could be loaded, it would not be loaded. Therefore, we can add a parent to the ClassLoader by modifying the parent of the ClassLoader. Change the original BootClassLoader < -pathClassLoader structure to BootClassLoader < -dexClassLoader < -pathClassLoader, Adding DexClassLoader to ContainerActivity causes the system to find the correct implementation of ContainerActivity when it looks up the PathClassLoader.

So we had no choice but to make the Container dynamic, and also used a unique reflection to change the private variable in this dynamic. It’s worth acknowledging that Shadow does have this reflection in all open-source code. It’s a bit of a contradiction to Shadow’s claim of zero reflection. The argument here is that zero reflection is in contrast to whether or not traditional plug-in frameworks use reflection when dynamically loading components such as activities. The dynamic of Containers, or even the Dynamic layer of Shadow, is not a necessary part of solving the same problem as other plug-in frameworks. In particular, the dynamic of the Container is optional.

The parent domain of a ClassLoader is not a private API, or even Android code, but JDK code. And the implementation of this reflection does not require hard coding of the word “parent” because the getParent method allows us to determine the parent field by run-time comparison. So, this reflection implementation is relatively safe, and we’ve actually been running it online for three years, and we’ve never seen a failure.

The dynamics of containers, while arguably unnecessary, do have benefits. With the dynamic of the Container, there is no need to implement all methods on the Container that require Override at once. It can be added as the business needs it.

About the Container of dynamic, can see concrete com. Tencent. Shadow. Dynamic. Host. DynamicRuntime the implementation of a class.

In addition, the Runtime contains containers, but really only containers need to be dynamic in this way. Other classes in the Runtime are grouped together because of simplified implementation, and other classes can be loaded conventionally, just above the PluginClassLoader.

Practical experience with dynamic loading interface implementation

Finally, a bit of experience about loading interface implementations. We wrote in the Shadow of a ApkClassLoader class that encapsulates the com. Tencent. Shadow. Dynamic. Host. ApkClassLoader# getInterface generic method, interface type instances can be obtained directly. Note that this method does not support constructors that hold parameters. If you look at our code history, you can find the implementation with the parameter version, but it was eventually deleted. Because I found that passing a set of parameter type class tokens is not associated with implementing class constructors at compile time. If the constructor of the implementation class changes its argument list, the call to getInterface will not be detected at compile time. Will wait until runtime to throw an exception that cannot find the constructor for that argument list. So I changed it to define a factory interface, where getInterface always takes a factory interface and then builds an object with parameters through the factory interface. In this way, changes to the construction parameter list of the implementation class can be detected at compile time.

Can look at the Shadow in the code com. Tencent. Shadow. Dynamic. Host. LoaderFactory and com., tencent. Shadow. Dynamic. Host. ManagerFactory to analyze I tell the difference.

conclusion

Shadow dynamically loads all parts of the plug-in framework defined by us, so that the problems of the plug-in framework itself can be dynamically repaired and the plug-in framework becomes a part of the plug-in package, avoiding the problem that plug-ins need to adapt to different versions of the plug-in framework.

This feature is more important in practice than a hack-free implementation. Because it allows us to replace hundreds of reflection implementations with hack-free Shadow frameworks without even changing a single line of host code, without even following the host version. This allows us to make this switch without considering the maintenance of the old framework.