Article source: studyidea.cn/java-hotswa…
One, foreword
One afternoon when I was poking around, my little sister came over and asked for help, saying that she needed to change the mock application in the test environment. But the application won’t find the source code for a while. Arthas is able to update application code in a hot way, decompilating the application code and adding the logic that needs to be changed by following the steps on the Internet. The test sister is very satisfied with this, and said that next time I will mention fewer bugs.
Heh heh, I have been curious about the principle behind hot update before, so I take this opportunity to study the principle of hot update.
Arthas thermal update
Let’s take a look at how Arthas is being thermally updated.
Details: Alibaba Arthas Practice — JAD/MC/Re-define online hot update dragon
Suppose we now have a HelloService class, the logic is as follows, and now we use Arthas hot update code to output Hello Arthas.
public class HelloService {
public static void main(String[] args) throws InterruptedException {
while (true){
TimeUnit.SECONDS.sleep(1); hello(); }}public static void hello(a){
System.out.println("hello world"); }}Copy the code2.1. Jad decompiles the code
First, run jad to decompile the class file to obtain the source code. Run the following command:
jad --source-only com.andyxh.HelloService > /tmp/HelloService.java
Copy the code2.2. Modify the decompiled code
Once you have the source code, use a text editing tool like VIM to edit the source code and add the logic that needs to be changed.
2.3, findClassLoader
Then use the sc command to find the ClassLoader to load the modified class, run the following command:
$ sc -d com.andyxh.HelloService | grep classLoaderHash
classLoaderHash 4f8e5cde
Copy the codeRunning here will give you the ClassLoader hash.
2.4,mcMemory compiled source code
Compile the saved source code using the MC command to generate the final class file.
$ mc -c 4f8e5cde /tmp/HelloService.java -d /tmp
Memory compiler output:
/tmp/com/andyxh/HelloService.class
Affect(row-cnt:1) cost in 463 ms.
Copy the code2.5, Re-define hot update code
Run re-define:
$ redefine /tmp/com/andyxh/HelloService.class
redefine success, size: 1
Copy the codeAfter successful hot update, the program output is as follows:
In general, we will have the source code locally, so we can skip this step further. We can change the code on our IDE and compile the class file first. All we need to do is run re-define. In other words, it’s just re-define.
Three, Instrumentation and attach mechanism
Arthas hot update looks like a magic feature that actually relies on JDK apis called instrument API and Attach API.
3.1 Instrumentation
Java Instrumentation is the interface provided after JDK5. Using this set of interfaces, we can get information about the running JVM and use this information to build monitoring programs to detect the JVM. In addition, most importantly we can replace and modify the class, so that hot update.
Instrumentation can be used in two ways, one is pre-main, in which the Instrumentation program needs to be specified in the vm parameters, and then the program will be modified or replaced before starting. The usage is as follows:
java -javaagent:jar Instrumentation_jar -jar xxx.jar
Copy the codeIf this startup mode sounds familiar, take a closer look at the IDEA run output window.
Many other application monitoring tools, such as Zipkin, Pinpoint, Skywalking.
This method can only take effect before the application is started, which has some limitations.
JDK6 improves this situation by adding agent-main mode. We can run Instrumentation after the application is started. Once started, we can’t make any changes until we connect to the corresponding application, which is where we need to provide the Attach API using Java.
3.2 Attach the API
The Attach API is located in the Tools. jar package and can be used to connect to the target JVM. The Attach API is very simple, with only two main classes inside, VirtualMachine and VirtualMachineDescriptor.
VirtualMachine represents an instance of the JVM, which provides the ATTACH method so that we can connect to the target JVM.
VirtualMachine vm = VirtualMachine.attach(pid);
Copy the codeVirtualMachineDescriptor is a container class that describes virtual machines. From this instance we can get the JVM PID(process ID). This instance is mainly obtained through the VirtualMachine#list method.
for (VirtualMachineDescriptor descriptor : VirtualMachine.list()){
System.out.println(descriptor.id());
}
Copy the codeAfter introducing the related principles of hot update, we can use the above API to realize hot update function.
Fourth, to achieve hot update function
We use Instrumentation agent-main here.
4.1. Agent-main is implemented
First you need to write a class that contains the following two methods:
public static void agentmain (String agentArgs, Instrumentation inst); [1]
public static void agentmain (String agentArgs); [2]
Copy the codeYou only need to implement one of the above methods. If both are implemented, the priority of [1] is greater than that of [2].
Instrumentation#redefineClasses. This method will use the new class to replace the current running class. So we are done modifying the class.
public class AgentMain {
/ * * * *@paramAgentArgs External argument, similar to main function args *@param inst
*/
public static void agentmain(String agentArgs, Instrumentation inst) {
// Get external parameters from agentArgs
System.out.println("Start hot updating code");
// The class file path will be passed in here
String path = agentArgs;
try {
// Read the class file bytecode
RandomAccessFile f = new RandomAccessFile(path, "r");
final byte[] bytes = new byte[(int) f.length()];
f.readFully(bytes);
// Use the ASM framework to get the class name
final String clazzName = readClassName(bytes);
// The inst.getAllLoadedClasses method will fetch all loaded classes
for (Class clazz : inst.getAllLoadedClasses()) {
// The match needs to replace the class
if (clazz.getName().equals(clazzName)) {
ClassDefinition definition = new ClassDefinition(clazz, bytes);
// Replace the current system using class with the specified classinst.redefineClasses(definition); }}}catch (UnmodifiableClassException | IOException | ClassNotFoundException e) {
System.out.println("Hot update data failed"); }}/** * use ASM to read the class name **@param bytes
* @return* /
private static String readClassName(final byte[] bytes) {
return new ClassReader(bytes).getClassName().replace("/"."."); }}Copy the codeAfter completing the code, we also need to write the following attributes to the JAR package manifest.
Agent-main: com.andyxh.AgentMain # re-define Class: true # re-define Class: true # re-define Class: trueCopy the codeWe use maven-assembly-plugin to write the above properties to the file.
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<! < jar name >
<finalName>hotswap-jdk</finalName>
<appendAssemblyId>false</appendAssemblyId>
<descriptorRefs>
<! Package the project dependency jar together -->
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifestEntries>
<! -- Specify class name -->
<Agent-Class>
com.andyxh.AgentMain
</Agent-Class>
<Can-Redefine-Classes>
true
</Can-Redefine-Classes>
</manifestEntries>
<manifest>
<! -- specify the mian class name, which will be used below -->
<mainClass>com.andyxh.JvmAttachMain</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <! -- this is used for inheritance merges -->
<phase>package</phase> <! -- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
Copy the codeAt this point we complete the hot update main code, then use the Attach API to connect to the target VM and trigger the hot update code.
public class JvmAttachMain {
public static void main(String[] args) throws IOException, AttachNotSupportedException, AgentLoadException, AgentInitializationException {
// Attach JVM PID and class path
if(args==null||args.length<2){
System.out.println("Please enter the necessary parameters. The first parameter is PID and the second parameter is class absolute path.");
return;
}
String pid=args[0];
String classPath=args[1];
System.out.println("Currently hot update JVM PID is required"+pid);
System.out.println("Change the class absolute path to"+classPath);
// Get the current jar path
URL jarUrl=JvmAttachMain.class.getProtectionDomain().getCodeSource().getLocation();
String jarPath=jarUrl.getPath();
System.out.println("The current hot update tool JAR path is"+jarPath);
VirtualMachine vm = VirtualMachine.attach(pid);//7997 is the PID of the JVM process to be bound
// Run the final AgentMain methodvm.loadAgent(jarPath, classPath); }}Copy the codeIn this startup class, we finally call VirtualMachine#loadAgent, and the JVM will replace the running class with the incoming class file using the AgentMain method above.
4.2, run,
Here we continue with the previous example, but add a method to get the JVM running process ID.
public class HelloService {
public static void main(String[] args) throws InterruptedException {
System.out.println(getPid());
while (true){
TimeUnit.SECONDS.sleep(1); hello(); }}public static void hello(a){
System.out.println("hello world");
}
/** * Get the current running JVM PID *@return* /
private static String getPid(a) {
// get name representing the running Java virtual machine.
String name = ManagementFactory.getRuntimeMXBean().getName();
System.out.println(name);
// get pid
return name.split("@") [0]; }}Copy the codeFirst run HelloService, get the current PID, then copy HelloService code to another project, modify hello method output Hello Agent, recompile to generate a new class file.
Finally, run the generated JAR package on the command line.
The HelloService output looks like this:
Source address: https://github.com/9526xu/hotswap-example
4.3 debugging skills
For normal applications, we can Debug programs directly in IDE mode, but the above programs cannot use Debug directly. The program ran into a lot of problems at the beginning, but had no choice but to choose the most primitive method, print error log. When I looked at arthas’s documentation, I found the previous article about debugging programs using the IDEA Remote Debug mode.
First we need to add the following parameters to the HelloService JVM parameter:
-Xrunjdwp:transport=dt_socket,server=y,address=8001
Copy the codeThe program will block until the remote debugger connects to port 8001, and the output is as follows:
Then add a remote debugging in agent-main.
The parameters in the figure are as follows:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8001
Copy the codeCreate a breakpoint in agent-main, run remote debugging, and the HelloService program will be started.
Finally, run the agent-main program in the command line window, and the remote debugging will be paused at the corresponding breakpoint. The following debugging is not described as normal Debug mode.
4.4. Related issues
Since the Attach API is located in tools.jar, the tools.jar is not in the same location as the JDK jar before JDK8, so the compile and run process may not find the JAR, resulting in an error.
If Maven is compiled and run using JDK9, don’t worry.
Maven compilation Problems
The following errors may occur during maven compilation.
The solution is to add tools.jar to the POM.
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1.6</version>
<systemPath>${java.home}/.. /lib/tools.jar</systemPath>
</dependency>
Copy the codeOr use the following dependencies.
<dependency>
<groupId>com.github.olivergondza</groupId>
<artifactId>maven-jdk-tools-wrapper</artifactId>
<version>0.1</version>
<scope>provided</scope>
<optional>true</optional>
</dependency>
Copy the codeThe tools.jar process could not be found
To run the program thrown when the Java. Lang. NoClassDefFoundError, main reason is system did not find the tools. The jar.
-xbootclasspath /a:${java_HOME}/lib/tools.jar
4.5. Hot updates have some limitations
Instrumentation#redefineClasses not all modification hot updates will be successful, there are currently some limitations to using Instrumentation#redefineClasses. We can only modify the internal logic of methods, attribute values, etc., and cannot add or delete methods or fields, nor can we change the signature or inheritance relationship of methods.
Five, the eggs
After writing the hot update code, I received an email from the system reminding me that XXX bug needs to be fixed. Well, we agreed not to mention bugs o(╥﹏╥)o.
Six, help
1. Explore Java hot deployment in depth. 2
Welcome to pay attention to my public account: procedures to get daily dry goods push. If you are interested in my topics, you can also follow my blog: studyidea.cn
