preface
I remember that when I first learned Java, I came into contact with the feature of Reflection provided by Java as soon as I finished learning the basis of syntax. Although this is a very basic knowledge point now, I was definitely excited at that time, and instantly felt that I was separated from the team of “Java beginners”. With the broadening of my working experience, I learned many similar things that made me excited, such as using skills unsafe. Unsafe was definitely one of them.
Sun.misc.Unsafe is a utility class native to the JDK, which contains many cool operations in the Java language, such as memory allocation and reclamation, CAS operations, class instantiation, and memory barriers. As its name implies, the operations it provides are also dangerous because it can manipulate memory directly and make low-level system calls. Unsafe plays an important role in extending the expressive power of the Java language and making it easier to implement core library functions in higher-level (Java layer) code that would otherwise be implemented in lower-level (C layer) code.
From JDK9 onwards, the limitations of Java’s modular design made sun.misc.unsafe inaccessible to modules that are not standard libraries. In JDK8, however, you can still play with Unsafe directly, and if you don’t learn, you probably won’t be able to in the future.
The use of Unsafe
Unsafe is not designed to be called by ordinary developers, so you can’t instantiate an Unsafe object using either a new or factory method. Instead, you can get a reflected Unsafe instance:
public static final Unsafe unsafe = getUnsafe();
static sun.misc.Unsafe getUnsafe(a) {
try {
Field field = Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
return (Unsafe) field.get(null);
} catch (Exception e) {
throw newRuntimeException(e); }}Copy the codeOnce you get it, you can use the global singleton to do whatever you want.
Functions overview
I borrowed the picture from the Internet. The image above, featured by Unsafe, looks pretty comprehensive. Addressing Unsafe, the article would have been too long and the format would have been confusing. I’m going to talk about the application techniques of Unsafe from a practical perspective, combining my experience in projects and competitions.
Memory allocation & access
Java can also manipulate memory directly, like C++, using Unsafe. Let’s look at an example of a ByteBuffer. We will create a 16-byte memory space that will write and read four ints in sequence.
public static void testByteBuffer(a) {
ByteBuffer directBuffer = ByteBuffer.allocateDirect(16);
directBuffer.putInt(1);
directBuffer.putInt(2);
directBuffer.putInt(3);
directBuffer.putInt(4);
directBuffer.flip();
System.out.println(directBuffer.getInt());
System.out.println(directBuffer.getInt());
System.out.println(directBuffer.getInt());
System.out.println(directBuffer.getInt());
}
Copy the codeThose familiar with NIO operations will be familiar with the above example, which is very basic and standard memory usage. How does Unsafe achieve the same effect?
public static void testUnsafe0(a) {
Unsafe unsafe = Util.unsafe;
long address = unsafe.allocateMemory(16);
unsafe.putInt(address, 1);
unsafe.putInt(address + 4.2);
unsafe.putInt(address + 8.3);
unsafe.putInt(address + 12.4);
System.out.println(unsafe.getInt(address));
System.out.println(unsafe.getInt(address + 4));
System.out.println(unsafe.getInt(address + 8));
System.out.println(unsafe.getInt(address + 12));
}
Copy the codeThe output of the two codes is the same:
1
2
3
4
Copy the codeThe following is a look at the Unsafe apis used:
public native long allocateMemory(long var1);
Copy the codeThe native method allocates out-of-heap memory and returns a value of type long, the first address of the memory, which can be used as an input parameter to other Unsafe apis. If you look at DirectByteBuffer’s source code, it’s wrapped internally using Unsafe. Speaking of DirectByteBuffer, here, additional ByteBuffer. AllocateDirect distribution outside the heap memory to be – XX: MaxDirectMemorySize restrictions, and Unsafe distribution outside the heap memory is not limited, And, of course, it’s not bound by -xmx. If you’re participating in a competition and you’re inspired by it, you can type “I get it” on the public screen.
Looking at the other two apis, putInt and getInt, you should realize that there must be other apis for byte manipulation, such as putByte/putShort/putLong, and of course put and get also come in pairs. It is recommended that these apis be used in pairs, otherwise parsing may fail due to byte order problems. Here’s an example:
public static void testUnsafe1(a) {
ByteBuffer directBuffer = ByteBuffer.allocateDirect(4);
long directBufferAddress = ((DirectBuffer)directBuffer).address();
System.out.println("Unsafe.putInt(1)");
Util.unsafe.putInt(directBufferAddress, 1);
System.out.println("Unsafe.getInt() == " + Util.unsafe.getInt(directBufferAddress));
directBuffer.position(0);
directBuffer.limit(4);
System.out.println("ByteBuffer.getInt() == " + directBuffer.getInt());
directBuffer.position(0);
directBuffer.limit(4);
System.out.println("ByteBuffer.getInt() reverseBytes == " + Integer.reverseBytes(directBuffer.getInt()));
}
Copy the codeThe output is as follows:
Unsafe.putInt(1)
Unsafe.getInt() == 1
ByteBuffer.getInt() == 16777216
ByteBuffer.getInt() reverseBytes == 1
Copy the codeEven if Unsafe is used for putInt and ByteBuffer for getInt, the result is not as expected and requires byte order changes to recover the result. The underlying reason for this is that ByteBuffer internally determines the current operating system’s byte order. For multibyte data types like INT, my test machine used large enended storage, whereas Unsafe stores small short order by default. If in doubt, use the write and read apis to avoid byte order problems. For those of you who are not familiar with byte order, please refer to another article of mine called “What the hell is byte order”.
Memory replication
Memory replication is still a common requirement in practical application scenarios. For example, in the last article, when the memory in the heap is written to disk, it needs to be copied to the out-of-heap memory first. For example, when we do memory aggregation, we need to buffer some data, and memory replication is also involved. You can also use ByteBuffer or set/get, but it’s not as efficient as native. Unsafe provides native methods for copying memory, from in-heap to in-heap, out of heap to out of heap, out of heap and in-heap to each other, everywhere.
public native void copyMemory(Object src, long offset, Object dst ,long dstOffset, long size);
Copy the codeFor memory in the heap, we can pass SRC the first address of the array of objects and specify offset as the offset of the corresponding array type. We can obtain the offset of the object stored in the heap by using arrayBaseOffset
public native int arrayBaseOffset(Class
var1);
Copy the codeFor example, to obtain the fixed offset of byte[], you can do this: unsafe.arrayBaseoffset (byte[].class)
For out-of-heap memory, it’s a little more intuitive, with DST set to NULL and dstOffset set to Unsafe.
Example code for copying in-heap memory to out-of-heap memory:
public static void unsafeCopyMemory(a) {
ByteBuffer heapBuffer = ByteBuffer.allocate(4);
ByteBuffer directBuffer = ByteBuffer.allocateDirect(4);
heapBuffer.putInt(1234);
long address = ((DirectBuffer)directBuffer).address();
Util.unsafe.copyMemory(heapBuffer.array(), 16.null, address, 4);
directBuffer.position(0);
directBuffer.limit(4);
System.out.println(directBuffer.getInt());
}
Copy the codeIn practice, most bytebuffer-related source code uses the copyMemory method when it comes to memory copying.
Unconventionally instantiate objects
Before MODULarizing JDK9, there were two common practices if you didn’t want some classes to be open to other users or to be instantiated arbitrarily (the singleton pattern)
Case one: Privatize the constructor
public class PrivateConstructorFoo {
private PrivateConstructorFoo(a) {
System.out.println("constructor method is invoked");
}
public void hello(a) {
System.out.println("hello world"); }}Copy the codeIf you want to instantiate this object, the first thing that comes to mind is probably reflection creation
public static void reflectConstruction(a) {
PrivateConstructorFoo privateConstructorFoo = PrivateConstructorFoo.class.newInstance();
privateConstructorFoo.hello();
}
Copy the codeAs expected, we got an exception
java.lang.IllegalAccessException: Class io.openmessaging.Main can not access a member of class moe.cnkirito.PrivateConstructorFoo with modifiers "private"
Copy the codeWith a few tweaks, call the constructor to create the instance
public static void reflectConstruction2(a) {
Constructor<PrivateConstructorFoo> constructor = PrivateConstructorFoo.class.getDeclaredConstructor();
constructor.setAccessible(true);
PrivateConstructorFoo privateConstructorFoo = constructor.newInstance();
privateConstructorFoo.hello();
}
Copy the codeIt works! The output is as follows:
constructor method is invoked
hello world
Copy the codeBroadening also provides the allocateInstance method, of course
public native Object allocateInstance(Class
var1) throws InstantiationException;
Copy the codeIt can also be instantiated, and it’s much more intuitive
public static void allocateInstance(a) throws InstantiationException {
PrivateConstructorFoo privateConstructorFoo = (PrivateConstructorFoo) Util.unsafe.allocateInstance(PrivateConstructorFoo.class);
privateConstructorFoo.hello();
}
Copy the codeThe same works! The output is as follows:
hello world
Copy the codeNotice one detail here: allocateInstance does not trigger the constructor.
Case 2: Package Level instance
package moe.cnkirito;
class PackageFoo {
public void hello(a) {
System.out.println("hello world"); }}Copy the codeNote that I have defined a package-level accessible object, PackageFoo, that is only accessible to classes under the moe.cnkirito package.
Let’s also try reflection first
package com.bellamm;
public static void reflectConstruction(a) { Class<? > aClass = Class.forName("moe.cnkirito.PackageFoo");
aClass.newInstance();
}
Copy the codeGot the expected error:
java.lang.IllegalAccessException: Class io.openmessaging.Main can not access a member of class moe.cnkirito.PackageFoo with modifiers ""
Copy the codeWhat about Unsafe?
package com.bellamm;
public static void allocateInstance(a) throws Exception{ Class<? > fooClass = Class.forName("moe.cnkirito.PackageFoo");
Object foo = Util.unsafe.allocateInstance(fooClass);
Method helloMethod = fooClass.getDeclaredMethod("hello");
helloMethod.setAccessible(true);
helloMethod.invoke(foo);
}
Copy the codeUnder com.bellamm, we can’t even define the PackageFoo class at compile time. Instead, we need to get the moe.cnkirito.PackageFoo method at runtime through reflection. Successfully output Hello World.
Having spent so much time experimenting with the Unsafe#allocateInstance, as well as addressing the Unsafe solution, we needed a real-world application to demonstrate the value of Unsafe#allocateInstance. Let me briefly list two scenarios:
- Serialization frameworks that cannot create objects using reflection can also try using Unsafe as underhand logic.
- Obtain package level protection of the class, with the help of reflection mechanism, can magic change some source code implementation or call some native methods, this method should be used with caution, not recommended in production.
Example code: Dynamically modify the out-of-heap memory limit to override the JVM startup parameter: -xx :MaxDirectMemorySize
private void hackMaxDirectMemorySize(a) {
try {
Field directMemoryField = VM.class.getDeclaredField("directMemory");
directMemoryField.setAccessible(true);
directMemoryField.set(new VM(), 8L * 1024 * 1024 * 1024);
Object bits = Util.unsafe.allocateInstance(Class.forName("java.nio.Bits"));
Field maxMemory = bits.getClass().getDeclaredField("maxMemory");
maxMemory.setAccessible(true);
maxMemory.set(bits, 8L * 1024 * 1024 * 1024);
} catch (Exception e) {
throw new RuntimeException(e);
}
System.out.println(VM.maxDirectMemory());
}
Copy the codeconclusion
Looking Unsafe, I’d like to introduce these three Unsafe examples, which I personally think are most commonly used.
Unsafe, even people who know how to use it know not to use it. If not, it’s better to know that Java has this mechanism than not. Of course, this article also covers some practical scenarios where Unsafe might be necessary, but it’s mostly in the underlying source code.
If there are readers want to see more SAO operation, welcome to forward this article, read 1500, continue to add a more, a key three even, this time certain.