sequence
This article focuses on -xx :MaxDirectMemorySize for the JVM
-XX:MaxDirectMemorySize
-xx :MaxDirectMemorySize= SIZE Set the maximum size of New I/O(java.niO) direct-buffer allocations, size can be k/K, M /M, or g/ g. If this parameter is not set, the default is 0, meaning that the JVM automatically selects maximum size allocations for NIO direct-buffer
System.initPhase1
java.base/java/lang/System.java
public final class System {
/* Register the natives via the static initializer.
*
* VM will invoke the initializeSystemClass method to complete
* the initialization for this class separated from clinit.
* Note that to use properties set by the VM, see the constraints
* described in the initializeSystemClass method.
*/
private static native void registerNatives();
static {
registerNatives();
}
/** Don't let anyone instantiate this class */
private System() {
}
/**
* Initialize the system class. Called after thread initialization.
*/
private static void initPhase1() {
// VM might invoke JNU_NewStringPlatform() to set those encoding
// sensitive properties (user.home, user.name, boot.class.path, etc.)
// during "props" initialization.
// The charset is initialized in System.c and does not depend on the Properties.
Map<String, String> tempProps = SystemProps.initProperties();
VersionProps.init(tempProps);
// There are certain system configurations that may be controlled by
// VM options such as the maximum amount of direct memory and
// Integer cache size used to support the object identity semantics
// of autoboxing. Typically, the library will obtain these values
// from the properties set by the VM. If the properties are for
// internal implementation use only, these properties should be
// masked from the system properties.
//
// Save a private copy of the system properties object that
// can only be accessed by the internal implementation.
VM.saveProperties(tempProps);
props = createProperties(tempProps);
StaticProperty.javaHome(); // Load StaticProperty to cache the property values
lineSeparator = props.getProperty("line.separator");
FileInputStream fdIn = new FileInputStream(FileDescriptor.in);
FileOutputStream fdOut = new FileOutputStream(FileDescriptor.out);
FileOutputStream fdErr = new FileOutputStream(FileDescriptor.err);
setIn0(new BufferedInputStream(fdIn));
setOut0(newPrintStream(fdOut, props.getProperty("sun.stdout.encoding")));
setErr0(newPrintStream(fdErr, props.getProperty("sun.stderr.encoding")));
// Setup Java signal handlers for HUP, TERM, and INT (where available).
Terminator.setup();
// Initialize any miscellaneous operating system settings that need to be
// set for the class libraries. Currently this is no-op everywhere except
// for Windows where the process-wide error mode is set before the java.io
// classes are used.
VM.initializeOSEnvironment();
// The main thread is not added to its thread group in the same
// way as other threads; we must do it ourselves here.
Thread current = Thread.currentThread();
current.getThreadGroup().add(current);
// register shared secrets
setJavaLangAccess();
// Subsystems that are invoked during initialization can invoke
// VM.isBooted() in order to avoid doing things that should
// wait until the VM is fully initialized. The initialization level
// is incremented from 0 to 1 here to indicate the first phase of
// initialization has completed.
// IMPORTANT: Ensure that this remains the last initialization action!
VM.initLevel(1);
}
//......
}
Copy the codeThe initPhase1 method of the System calls the vm. saveProperties(tempProps) method to hold a copy of the System configuration for internal implementation use; Which is suitable for SystemProps tempProps. InitProperties ()
jvm.cpp
hotspot/share/prims/jvm.cpp
// Convert the -XX:MaxDirectMemorySize= command line flag
// to the sun.nio.MaxDirectMemorySize property.
// Do this after setting user properties to prevent people
// from setting the value with a -D option, as requested.
// Leave empty if not supplied
if(! FLAG_IS_DEFAULT(MaxDirectMemorySize)) { char as_chars[256]; jio_snprintf(as_chars, sizeof(as_chars), JULONG_FORMAT, MaxDirectMemorySize); Handle key_str = java_lang_String::create_from_platform_dependent_str("sun.nio.MaxDirectMemorySize", CHECK_NULL);
Handle value_str = java_lang_String::create_from_platform_dependent_str(as_chars, CHECK_NULL);
result_h->obj_at_put(ndx * 2, key_str());
result_h->obj_at_put(ndx * 2 + 1, value_str());
ndx++;
}
Copy the codeJVM. CPP is a piece of code is used to put – XX: MaxDirectMemorySize command parameter transformation for key for sun. Nio. MaxDirectMemorySize properties
VM.saveProperties
java.base/jdk/internal/misc/VM.java
public class VM { // the init level when the VM is fully initialized private static final int JAVA_LANG_SYSTEM_INITED = 1; private static final int MODULE_SYSTEM_INITED = 2; private static final int SYSTEM_LOADER_INITIALIZING = 3; private static final int SYSTEM_BOOTED = 4; private static final int SYSTEM_SHUTDOWN = 5; // 0, 1, 2... private static volatile int initLevel; private static final Object lock = new Object(); / /... // A user-settable upperlimit on the maximum amount of allocatable direct
// buffer memory. This value may be changed during VM initialization if
// "java" is launched with "-XX:MaxDirectMemorySize=<size>".
//
// The initial value of this field is arbitrary; during JRE initialization
// it will be reset to the value specified on the command line, if any,
// otherwise to Runtime.getRuntime().maxMemory().
//
private static long directMemory = 64 * 1024 * 1024;
// Returns the maximum amount of allocatable direct buffer memory.
// The directMemory variable is initialized during system initialization
// in the saveAndRemoveProperties method.
//
public static long maxDirectMemory() {
returndirectMemory; } / /... // Save a private copy of the system properties and remove // the system properties that are not intendedfor public access.
//
// This method can only be invoked during system initialization.
public static void saveProperties(Map<String, String> props) {
if(initLevel() ! = 0) throw new IllegalStateException("Wrong init level");
// only main thread is running at this time, so savedProps and
// its content will be correctly published to threads started later
if (savedProps == null) {
savedProps = props;
}
// Set the maximum amount of direct memory. This value is controlled
// by the vm option -XX:MaxDirectMemorySize=<size>.
// The maximum amount of allocatable direct buffer memory (in bytes)
// from the system property sun.nio.MaxDirectMemorySize set by the VM.
// If not set or set to -1, the max memory will be used
// The system property will be removed.
String s = props.get("sun.nio.MaxDirectMemorySize");
if (s == null || s.isEmpty() || s.equals("1")) {
// -XX:MaxDirectMemorySize not given, take default
directMemory = Runtime.getRuntime().maxMemory();
} else {
long l = Long.parseLong(s);
if (l > -1)
directMemory = l;
}
// Check if direct buffers should be page aligned
s = props.get("sun.nio.PageAlignDirectMemory");
if ("true".equals(s))
pageAlignDirectMemory = true; } / /... }Copy the codeVM saveProperties methods read sun. Nio. MaxDirectMemorySize attribute, if is null or is empty or is 1, then the Settings for the Runtime. GetRuntime (). MaxMemory (); If MaxDirectMemorySize is set and the value is greater than -1, use this value as the directMemory value. The VM’s maxDirectMemory method returns the value of directMemory
Gets the value of maxDirectMemory
The instance
public BufferPoolMXBean getDirectBufferPoolMBean() {return ManagementFactory.getPlatformMXBeans(BufferPoolMXBean.class)
.stream()
.filter(e -> e.getName().equals("direct"))
.findFirst()
.orElseThrow();
}
public JavaNioAccess.BufferPool getNioBufferPool() {return SharedSecrets.getJavaNioAccess().getDirectBufferPool();
}
/**
* -XX:MaxDirectMemorySize=60M
*/
@Test
public void testGetMaxDirectMemory(){ ByteBuffer.allocateDirect(25*1024*1024); Println (Runtime.geTruntime ().maxMemory() / 1024.0/1024.0); system.out.println (Runtime.getruntime ().maxMemory() / 1024.0/1024.0); System.out.println(vm.maxDirectMemory () / 1024.0/1024.0); Println (getDirectBufferPoolMBean().getTotalCapacity() / 1024.0/1024.0); system.out.println (getDirectBufferPoolMBean().getTotalCapacity() / 1024.0/1024.0); Println (getNioBufferPool().getTotalCapacity() / 1024.0/1024.0); system.out.println (getNioBufferPool().getTotalCapacity() / 1024.0/1024.0); }Copy the codeThe output
The following output is displayed:
4096.0
60.0
25.0
25.0
Copy the code- Due to the modularization of Java9, the VM was changed from the original sun.misc.VM to JDK. Code above default is unamed module, in order to use the JDK. Internal. Misc. Will need to use the VM – add – exports Java. Base/JDK. Internal. Misc = ALL – UNNAMED export it to the UNNAMED, That’s how it works
- Similarly java9 modular, SharedSecrets from sun. Misc. SharedSecrets changes to Java. The base under the module of the JDK. Internal. Access. SharedSecrets; To use – add – exports Java. Base/JDK. Internal access = ALL – UNNAMED export it to the UNNAMED, such ability can run
- The runtime.getruntime ().maxMemory() output is correct. BufferPoolMXBean and JavanioAccess. BufferPool getTotalCapacity return the directBuffer size, not the Max value
View directBuffer usage using the API
The instance
/**
* -XX:MaxDirectMemorySize=60M
*/
@Test
public void testGetDirectMemoryUsage(){ ByteBuffer.allocateDirect(30*1024*1024); System.out.println(getDirectBufferPoolMBean().getMemoryUsed() / 1024.0/1024.0); System.out.println(getNioBufferPool().getMemoryUsed() / 1024.0/1024.0); }Copy the codeThe output
The following output is displayed:
30.0
30.0
Copy the codeYou can see that the BufferPoolMXBean and javanioAccess. BufferPool getMemoryUsed return the directBuffer size
OOM
java.lang.OutOfMemoryError: Direct buffer memory
at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
Copy the codeIf the above ByteBuffer. AllocateDirect to allocate more than 60 m, is run throw OutOfMemoryError
Use NMT to view directBuffer usage
jcmd 3088 VM.native_memory scale=MB
3088:
Native Memory Tracking:
Total: reserved=5641MB, committed=399MB
- Java Heap (reserved=4096MB, committed=258MB)
(mmap: reserved=4096MB, committed=258MB)
- Class (reserved=1032MB, committed=6MB)
(classes # 1609)
( instance classes #1460, array classes #149)(mmap: reserved=1032MB, committed=5MB) ( Metadata: ) (reserved=8MB, committed=5MB) (used=3MB) (free=2MB) (waste=0MB =0.00%) (Class space:) (reserved=1024MB, committed=5MB) (used=3MB) (free=2MB) (waste=0MB =0.00%) (Class space:) (reserved=1024MB, Committed =1MB) (Used =0MB) (Free =0MB) (waste=0MB =0.00%) - Thread (Reserved =18MB, COMMITTED =18MB) (Thread)# 18)
(stack: reserved=18MB, committed=18MB)
- Code (reserved=242MB, committed=7MB)
(mmap: reserved=242MB, committed=7MB)
- GC (reserved=203MB, committed=60MB)
(malloc=18MB # 2443)
(mmap: reserved=185MB, committed=43MB)
- Internal (reserved=1MB, committed=1MB)
(malloc=1MB # 1257)
- Other (reserved=30MB, committed=30MB)
(malloc=30MB # 2)
- Symbol (reserved=1MB, committed=1MB)
(malloc=1MB # 13745)
- Shared class space (reserved=17MB, committed=17MB)
(mmap: reserved=17MB, committed=17MB)
Copy the codeFrom the Other part can see its value with the ByteBuffer. AllocateDirect value used is consistent, change ByteBuffer. AllocateDirect value view again, you can find Other parts change accordingly; Therefore, the Other part should reflect the size of direct memory usage
summary
- -xx :MaxDirectMemorySize=size
java.nio) Maximum size of direct-buffer allocations, units of size can be K /K, M /M, or G/g; If this parameter is not set, the default is 0, meaning that the JVM automatically selects maximum size allocations for NIO direct-buffer. From Java code. The base/JDK/internal/misc/VM. Can see the default is to take in the Java Runtime. GetRuntime (). MaxMemory () - Using JDK. Internal. Misc. VM. MaxDirectMemory () to get the maxDirectMemory value; Due to the modularization of Java9, the VM was changed from the original sun.misc.VM to JDK. Code above default is unamed module, in order to use the JDK. Internal. Misc. Will need to use the VM – add – exports Java. Base/JDK. Internal. Misc = ALL – UNNAMED export it to the UNNAMED, That’s how it works
- BufferPoolMXBean and JavaNioAccess. BufferPool (
Obtain this document by using SharedSecretsGetMemoryUsed to get the size of direct memory; After the java9 modular, SharedSecrets from sun. Misc. SharedSecrets changes to Java. The base module of the JDK. Internal. Access. SharedSecrets; To use – add – exports Java. Base/JDK. Internal access = ALL – UNNAMED export it to the UNNAMED, such ability can run
You can also use NMT to view direct Memory usage, which is included in the Other section
doc
- Default HotSpot Maximum Direct Memory Size
- JVM source code analysis out of the heap memory full interpretation
- Check the Case of Java out-of-heap memory growth
- The DirectByteBuffer memory overflowed. Procedure
- Default for XX:MaxDirectMemorySize
- java.lang.OutOfMemoryError: Direct buffer memory (with tcnative) #6813
- VM.java MaxDirectMemorySize
- Summary of understanding “-XX:MaxDirectMemorySize” setting.
- native-mem-tracking.md
- Java Platform, Standard Edition Tools Reference