In view of the type of OOM we are not familiar with, then summarize the various OOM situations and solutions. Make a list of OOM situations, then code them one by one and provide solutions.
1. The stack overflow – Java. Lang. OutOfMemoryError: Java heap space.
2. The stack overflow – Java. Lang. OutOfMemorryError.
3. The stack overflow – Java. Lang. StackOverFlowError.
4. The meta information overflow – Java. Lang. OutOfMemoryError: Metaspace.
5. Direct memory – Java. Lang. OutOfMemoryError: Direct buffer memory.
6. GC overrun – Java. Lang. OutOfMemoryError: GC overhead limit is exceeded.
0x01: Heap overflow exception. That is, objects in the heap can’t be reclaimed, the heap keeps growing, so it reaches the maximum heap, it’s full, so it’s out. Let’s go straight to the overflow code sample. The VM Options of idea should be set to -xmx100m, so that the maximum heap memory can be set to error.
package oom; import java.util.ArrayList; import java.util.List; import java.util.concurrent.TimeUnit; public class HeapOOM { static class OOMObject { } public static void main(String[] args) throws InterruptedException { List<OOMObject> list = new ArrayList<>(); while(true) { // TimeUnit.MILLISECONDS.sleep(1); list.add(new OOMObject()); }}}Copy the codeThe exception that runs is as follows, and the code comes straight out of the exception that we see.
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java -Xmx100m -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/huangqingshi/Downloads/flink-project/target/classes:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-java/1.7.2/flink-java-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-core/1.7.2/flink-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-annotations/1.7.2/flink-annotations-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-metrics-core/1.7.2/flink-metrics-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/huangqingshi/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/huangqingshi/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-compress/1.18/commons-compress-1.18.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-asm/5.0.4-5.0/flink-shaded-asm-5.0.4-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/Users/huangqingshi/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/force-shading/1.7.2/force-shading-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-streaming-java_2.11/1.7.2/flink-streaming-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-runtime_2.11/1.7.2/flink-runtime_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-queryable-state-client-java_2.11/1.7.2/flink-queryable-state-client-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-hadoop-fs/1.7.2/flink-hadoop-fs-1.7.2.jar:/Users/huangqingshi/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-netty/4.1.24.Final-5.0/flink-shaded-netty-4.1.24.Final-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-jackson/2.7.9-5.0/flink-shaded-jackson-2.7.9-5.0.jar:/Users/huangqingshi/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar:/Users/huangqingshi/.m2/repository/org/javassist/javassist/3.19.0-GA/javassist-3.19.0-GA.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.4.20/akka-actor_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/config/1.3.0/config-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.11/0.7.0/scala-java8-compat_2.11-0.7.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-stream_2.11/2.4.20/akka-stream_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/reactivestreams/reactive-streams/1.0.0/reactive-streams-1.0.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-protobuf_2.11/2.4.20/akka-protobuf_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.4.20/akka-slf4j_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/clapper/grizzled-slf4j_2.11/1.3.2/grizzled-slf4j_2.11-1.3.2.jar:/Users/huangqingshi/.m2/repository/com/github/scopt/scopt_2.11/3.5.0/scopt_2.11-3.5.0.jar:/Users/huangqingshi/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill_2.11/0.7.6/chill_2.11-0.7.6.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill-java/0.7.6/chill-java-0.7.6.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-clients_2.11/1.7.2/flink-clients_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-optimizer_2.11/1.7.2/flink-optimizer_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-guava/18.0-5.0/flink-shaded-guava-18.0-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.10_2.11/1.7.2/flink-connector-kafka-0.10_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.9_2.11/1.7.2/flink-connector-kafka-0.9_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-base_2.11/1.7.2/flink-connector-kafka-base_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/kafka/kafka-clients/0.10.2.1/kafka-clients-0.10.2.1.jar:/Users/huangqingshi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.7/slf4j-log4j12-1.7.7.jar:/Users/huangqingshi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/huangqingshi/.m2/repository/com/alibaba/fastjson/1.2.62/fastjson-1.2.62.jar:/Users/huangqingshi/.m2/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:/Users/huangqingshi/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/huangqingshi/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/huangqingshi/.m2/repository/org/checkerframework/checker-qual/2.8.1/checker-qual-2.8.1.jar:/Users/huangqingshi/.m2/repository/com/google/errorprone/error_prone_annotations/2.3.2/error_prone_annotations-2.3.2.jar:/Users/huangqingshi/.m2/repository/com/google/j2objc/j2objc-annotations/1.3/j2objc-annotations-1.3.jar:/Users/huangqingshi/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.18/animal-sniffer-annotations-1.18.jar:/Users/huangqingshi/.m2/repository/redis/clients/jedis/3.1.0/jedis-3.1.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-pool2/2.6.2/commons-pool2-2.6.2.jar:/Users/huangqingshi/.m2/repository/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar:/Users/huangqingshi/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar:/Users/huangqingshi/.m2/repository/com/alibaba/druid/1.1.20/druid-1.1.20.jar oom.HeapOOM
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3210)
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:261)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
at java.util.ArrayList.add(ArrayList.java:458)
at oom.HeapOOM.main(HeapOOM.java:21)
Process finished with exit code 1
Copy the codeIf you are careful, you will notice that a sleep is set in the code to see the structure and data graph of the heap memory. Turn on the hibernation code, then open the JDK’s built-in jConsole command, and after connecting, take a look at the overview diagram, which shows that the heap memory continues to grow.
Open the memory screen, take a look at the memory, and click the GC button. At this point, some classes will be reclaimed, but it will continue to grow, as shown in the figure below.
Click on the info TAB and have a look. After several GC collections, the amount of data in the class still doesn’t change much, indicating that no collection has taken place.
The solution to this situation is to find the problem point, analyze where there is a large number of classes that are not being collected, and then export the heap memory on the line using the JMAP command.
0x02: Looking at stack overflow, the following code is an infinite number of threads to create until no more threads can be created.
package oom; import java.util.concurrent.TimeUnit; /** * @Date 2020-07-18 */ public class StackOOM { public static void infiniteRun() { while(true) { Thread thread = new Thread(() -> { while (true) { try { TimeUnit.HOURS.sleep(1); } catch (InterruptedException e) { e.printStackTrace(); }}}); thread.start(); } } public static void main(String[] args) { infiniteRun(); }}Copy the codeIf we really need to create threads, we need to adjust the frame stack size -Xss512k, the default frame stack size is 1M, if set to smaller, can create more threads.
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java -Xss512k -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/huangqingshi/Downloads/flink-project/target/classes:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-java/1.7.2/flink-java-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-core/1.7.2/flink-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-annotations/1.7.2/flink-annotations-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-metrics-core/1.7.2/flink-metrics-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/huangqingshi/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/huangqingshi/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-compress/1.18/commons-compress-1.18.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-asm/5.0.4-5.0/flink-shaded-asm-5.0.4-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/Users/huangqingshi/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/force-shading/1.7.2/force-shading-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-streaming-java_2.11/1.7.2/flink-streaming-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-runtime_2.11/1.7.2/flink-runtime_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-queryable-state-client-java_2.11/1.7.2/flink-queryable-state-client-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-hadoop-fs/1.7.2/flink-hadoop-fs-1.7.2.jar:/Users/huangqingshi/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-netty/4.1.24.Final-5.0/flink-shaded-netty-4.1.24.Final-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-jackson/2.7.9-5.0/flink-shaded-jackson-2.7.9-5.0.jar:/Users/huangqingshi/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar:/Users/huangqingshi/.m2/repository/org/javassist/javassist/3.19.0-GA/javassist-3.19.0-GA.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.4.20/akka-actor_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/config/1.3.0/config-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.11/0.7.0/scala-java8-compat_2.11-0.7.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-stream_2.11/2.4.20/akka-stream_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/reactivestreams/reactive-streams/1.0.0/reactive-streams-1.0.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-protobuf_2.11/2.4.20/akka-protobuf_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.4.20/akka-slf4j_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/clapper/grizzled-slf4j_2.11/1.3.2/grizzled-slf4j_2.11-1.3.2.jar:/Users/huangqingshi/.m2/repository/com/github/scopt/scopt_2.11/3.5.0/scopt_2.11-3.5.0.jar:/Users/huangqingshi/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill_2.11/0.7.6/chill_2.11-0.7.6.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill-java/0.7.6/chill-java-0.7.6.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-clients_2.11/1.7.2/flink-clients_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-optimizer_2.11/1.7.2/flink-optimizer_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-guava/18.0-5.0/flink-shaded-guava-18.0-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.10_2.11/1.7.2/flink-connector-kafka-0.10_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.9_2.11/1.7.2/flink-connector-kafka-0.9_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-base_2.11/1.7.2/flink-connector-kafka-base_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/kafka/kafka-clients/0.10.2.1/kafka-clients-0.10.2.1.jar:/Users/huangqingshi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.7/slf4j-log4j12-1.7.7.jar:/Users/huangqingshi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/huangqingshi/.m2/repository/com/alibaba/fastjson/1.2.62/fastjson-1.2.62.jar:/Users/huangqingshi/.m2/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:/Users/huangqingshi/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/huangqingshi/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/huangqingshi/.m2/repository/org/checkerframework/checker-qual/2.8.1/checker-qual-2.8.1.jar:/Users/huangqingshi/.m2/repository/com/google/errorprone/error_prone_annotations/2.3.2/error_prone_annotations-2.3.2.jar:/Users/huangqingshi/.m2/repository/com/google/j2objc/j2objc-annotations/1.3/j2objc-annotations-1.3.jar:/Users/huangqingshi/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.18/animal-sniffer-annotations-1.18.jar:/Users/huangqingshi/.m2/repository/redis/clients/jedis/3.1.0/jedis-3.1.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-pool2/2.6.2/commons-pool2-2.6.2.jar:/Users/huangqingshi/.m2/repository/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar:/Users/huangqingshi/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar:/Users/huangqingshi/.m2/repository/com/alibaba/druid/1.1.20/druid-1.1.20.jar oom.StackOOM
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at oom.StackOOM.infiniteRun(StackOOM.java:24)
at oom.StackOOM.main(StackOOM.java:29)
Process finished with exit code 130 (interrupted by signal 2: SIGINT)
Copy the codeIf this is the case, and you need to know where many threads are being created, the online application needs to use the jstack command to export the current thread state into a file, and then upload the file to FastThread. IO for analysis.
0x03: look at another case of StackOverFlow, this is the case of StackOverFlow. The following is an endless recursive call.
package oom; /** * @Date 2020-07-18 */ public class StackOFE { public static void stackOverFlowErrorMethod() { stackOverFlowErrorMethod(); } public static void main(String[] args) { stackOverFlowErrorMethod(); }}Copy the codeEach time the program recurses, the program pushes the data, including the pointer, onto the stack. In this case, the frame stack needs to be larger to withstand more recursive calls. With -xss, the previous example needs to be set smaller to allocate more frames, this time a frame stack needs to record program data, so a larger value is needed.
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java -Xss2m -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/huangqingshi/Downloads/flink-project/target/classes:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-java/1.7.2/flink-java-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-core/1.7.2/flink-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-annotations/1.7.2/flink-annotations-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-metrics-core/1.7.2/flink-metrics-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/huangqingshi/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/huangqingshi/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-compress/1.18/commons-compress-1.18.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-asm/5.0.4-5.0/flink-shaded-asm-5.0.4-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/Users/huangqingshi/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/force-shading/1.7.2/force-shading-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-streaming-java_2.11/1.7.2/flink-streaming-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-runtime_2.11/1.7.2/flink-runtime_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-queryable-state-client-java_2.11/1.7.2/flink-queryable-state-client-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-hadoop-fs/1.7.2/flink-hadoop-fs-1.7.2.jar:/Users/huangqingshi/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-netty/4.1.24.Final-5.0/flink-shaded-netty-4.1.24.Final-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-jackson/2.7.9-5.0/flink-shaded-jackson-2.7.9-5.0.jar:/Users/huangqingshi/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar:/Users/huangqingshi/.m2/repository/org/javassist/javassist/3.19.0-GA/javassist-3.19.0-GA.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.4.20/akka-actor_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/config/1.3.0/config-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.11/0.7.0/scala-java8-compat_2.11-0.7.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-stream_2.11/2.4.20/akka-stream_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/reactivestreams/reactive-streams/1.0.0/reactive-streams-1.0.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-protobuf_2.11/2.4.20/akka-protobuf_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.4.20/akka-slf4j_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/clapper/grizzled-slf4j_2.11/1.3.2/grizzled-slf4j_2.11-1.3.2.jar:/Users/huangqingshi/.m2/repository/com/github/scopt/scopt_2.11/3.5.0/scopt_2.11-3.5.0.jar:/Users/huangqingshi/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill_2.11/0.7.6/chill_2.11-0.7.6.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill-java/0.7.6/chill-java-0.7.6.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-clients_2.11/1.7.2/flink-clients_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-optimizer_2.11/1.7.2/flink-optimizer_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-guava/18.0-5.0/flink-shaded-guava-18.0-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.10_2.11/1.7.2/flink-connector-kafka-0.10_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.9_2.11/1.7.2/flink-connector-kafka-0.9_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-base_2.11/1.7.2/flink-connector-kafka-base_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/kafka/kafka-clients/0.10.2.1/kafka-clients-0.10.2.1.jar:/Users/huangqingshi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.7/slf4j-log4j12-1.7.7.jar:/Users/huangqingshi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/huangqingshi/.m2/repository/com/alibaba/fastjson/1.2.62/fastjson-1.2.62.jar:/Users/huangqingshi/.m2/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:/Users/huangqingshi/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/huangqingshi/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/huangqingshi/.m2/repository/org/checkerframework/checker-qual/2.8.1/checker-qual-2.8.1.jar:/Users/huangqingshi/.m2/repository/com/google/errorprone/error_prone_annotations/2.3.2/error_prone_annotations-2.3.2.jar:/Users/huangqingshi/.m2/repository/com/google/j2objc/j2objc-annotations/1.3/j2objc-annotations-1.3.jar:/Users/huangqingshi/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.18/animal-sniffer-annotations-1.18.jar:/Users/huangqingshi/.m2/repository/redis/clients/jedis/3.1.0/jedis-3.1.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-pool2/2.6.2/commons-pool2-2.6.2.jar:/Users/huangqingshi/.m2/repository/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar:/Users/huangqingshi/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar:/Users/huangqingshi/.m2/repository/com/alibaba/druid/1.1.20/druid-1.1.20.jar oom.StackOFE
Exception in thread "main" java.lang.StackOverflowError
at oom.StackOFE.stackOverFlowErrorMethod(StackOFE.java:10)
at oom.StackOFE.stackOverFlowErrorMethod(StackOFE.java:10)
at oom.StackOFE.stackOverFlowErrorMethod(StackOFE.java:10)
at oom.StackOFE.stackOverFlowErrorMethod(StackOFE.java:10)
at oom.StackOFE.stackOverFlowErrorMethod(StackOFE.java:10)
at oom.StackOFE.stackOverFlowErrorMethod(StackOFE.java:10)
Copy the codeIn this case, the thread data needs to be exported to a file for analysis via JStack. Find the point at which recursion occurs. If recursion is the number of times the program needs to recurse, then increase the frame stack size to fit the program.
0x04: Metadata area overflow, metadata area is also called method area, which stores information about classes, constant pool, method descriptors, field descriptors. This area overflow occurs when a large number of classes are generated at runtime. -xx :MetaspaceSize=10M -xx :MaxMetaspaceSize=30M
package oom; import net.sf.cglib.proxy.Enhancer; import net.sf.cglib.proxy.MethodInterceptor; import net.sf.cglib.proxy.MethodProxy; import java.lang.reflect.Method; /** * @Date 2020-07-18 */ public class MetaspaceOOM { static class OOMObject{} public static void main(String[] args) { while (true) { Enhancer enhancer = new Enhancer(); enhancer.setSuperclass(OOMObject.class); enhancer.setUseCache(false); enhancer.setCallback(new MethodInterceptor() { public Object intercept(Object obj, Method method, Object[] args, MethodProxy proxy) throws Throwable { return proxy.invokeSuper(obj, args); }}); enhancer.create(); }}}Copy the codeThe result of the run is that the metadata information overflows. The reasons for this are as follows: generating a large number of classes through CBLIG causes the Meta information to be full; String. Intern () in JDK7 generates a lot of constant data. Load a lot of JSPS and dynamically generate JSP files. We need to adjust the size of the metadata space. If such exceptions still occur after the size is increased, we need to analyze where the overflow occurs and fix it.
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java -XX:MetaspaceSize=10M -XX:MaxMetaspaceSize=30M -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/huangqingshi/Downloads/flink-project/target/classes:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-java/1.7.2/flink-java-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-core/1.7.2/flink-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-annotations/1.7.2/flink-annotations-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-metrics-core/1.7.2/flink-metrics-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/huangqingshi/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/huangqingshi/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-compress/1.18/commons-compress-1.18.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-asm/5.0.4-5.0/flink-shaded-asm-5.0.4-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/Users/huangqingshi/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/force-shading/1.7.2/force-shading-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-streaming-java_2.11/1.7.2/flink-streaming-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-runtime_2.11/1.7.2/flink-runtime_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-queryable-state-client-java_2.11/1.7.2/flink-queryable-state-client-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-hadoop-fs/1.7.2/flink-hadoop-fs-1.7.2.jar:/Users/huangqingshi/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-netty/4.1.24.Final-5.0/flink-shaded-netty-4.1.24.Final-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-jackson/2.7.9-5.0/flink-shaded-jackson-2.7.9-5.0.jar:/Users/huangqingshi/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar:/Users/huangqingshi/.m2/repository/org/javassist/javassist/3.19.0-GA/javassist-3.19.0-GA.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.4.20/akka-actor_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/config/1.3.0/config-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.11/0.7.0/scala-java8-compat_2.11-0.7.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-stream_2.11/2.4.20/akka-stream_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/reactivestreams/reactive-streams/1.0.0/reactive-streams-1.0.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-protobuf_2.11/2.4.20/akka-protobuf_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.4.20/akka-slf4j_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/clapper/grizzled-slf4j_2.11/1.3.2/grizzled-slf4j_2.11-1.3.2.jar:/Users/huangqingshi/.m2/repository/com/github/scopt/scopt_2.11/3.5.0/scopt_2.11-3.5.0.jar:/Users/huangqingshi/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill_2.11/0.7.6/chill_2.11-0.7.6.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill-java/0.7.6/chill-java-0.7.6.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-clients_2.11/1.7.2/flink-clients_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-optimizer_2.11/1.7.2/flink-optimizer_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-guava/18.0-5.0/flink-shaded-guava-18.0-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.10_2.11/1.7.2/flink-connector-kafka-0.10_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.9_2.11/1.7.2/flink-connector-kafka-0.9_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-base_2.11/1.7.2/flink-connector-kafka-base_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/kafka/kafka-clients/0.10.2.1/kafka-clients-0.10.2.1.jar:/Users/huangqingshi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.7/slf4j-log4j12-1.7.7.jar:/Users/huangqingshi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/huangqingshi/.m2/repository/com/alibaba/fastjson/1.2.62/fastjson-1.2.62.jar:/Users/huangqingshi/.m2/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:/Users/huangqingshi/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/huangqingshi/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/huangqingshi/.m2/repository/org/checkerframework/checker-qual/2.8.1/checker-qual-2.8.1.jar:/Users/huangqingshi/.m2/repository/com/google/errorprone/error_prone_annotations/2.3.2/error_prone_annotations-2.3.2.jar:/Users/huangqingshi/.m2/repository/com/google/j2objc/j2objc-annotations/1.3/j2objc-annotations-1.3.jar:/Users/huangqingshi/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.18/animal-sniffer-annotations-1.18.jar:/Users/huangqingshi/.m2/repository/redis/clients/jedis/3.1.0/jedis-3.1.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-pool2/2.6.2/commons-pool2-2.6.2.jar:/Users/huangqingshi/.m2/repository/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar:/Users/huangqingshi/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar:/Users/huangqingshi/.m2/repository/com/alibaba/druid/1.1.20/druid-1.1.20.jar:/Users/huangqingshi/.m2/repository/cglib/cglib/3.2.5/cglib-3.2.5.jar:/Users/huangqingshi/.m2/repository/org/ow2/asm/asm/5.2/asm-5.2.jar:/Users/huangqingshi/.m2/repository/org/apache/ant/ant/1.9.6/ant-1.9.6.jar:/Users/huangqingshi/.m2/repository/org/apache/ant/ant-launcher/1.9.6/ant-launcher-1.9.6.jar oom.MetaspaceOOM
Exception in thread "main" java.lang.OutOfMemoryError: Metaspace
at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:345)
at net.sf.cglib.proxy.Enhancer.generate(Enhancer.java:492)
at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:114)
at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:291)
at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:480)
at net.sf.cglib.proxy.Enhancer.create(Enhancer.java:305)
at oom.MetaspaceOOM.main(MetaspaceOOM.java:28)
Process finished with exit code 1
Copy the code0x05: Direct memory overflow. In addition to using heap memory, it is possible to use direct memory, i.e., out-of-heap memory. NIO uses direct memory to improve performance by avoiding switching between Java Heap and Native Heap. By default, the size of direct memory is the same as that of peer memory. Out-of-heap memory is not limited by the JVM, but is limited by the size of the machine’s overall memory. The following code sets the maximum heap memory to 128MB, the direct memory to 100m, and then we allocate 1M at a time into the list.
-Xmx128m -XX:MaxDirectMemorySize=100M package oom; import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; /** * @Date 2020-07-18 */ public class DirectBufferOOM { public static void main(String[] args) { final int _1M = 1024 * 1024 * 1; List<ByteBuffer> buffers = new ArrayList<>(); int count = 1; while (true) { ByteBuffer byteBuffer = ByteBuffer.allocateDirect(_1M); buffers.add(byteBuffer); System.out.println(count++); }}}Copy the codeAt this point, when the output is 100 times, the next allocation will be OOM-Direct buffer memory.
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java -Xmx128m -XX:MaxDirectMemorySize=100M -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/huangqingshi/Downloads/flink-project/target/classes:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-java/1.7.2/flink-java-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-core/1.7.2/flink-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-annotations/1.7.2/flink-annotations-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-metrics-core/1.7.2/flink-metrics-core-1.7.2.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/huangqingshi/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/huangqingshi/.m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/huangqingshi/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-compress/1.18/commons-compress-1.18.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-asm/5.0.4-5.0/flink-shaded-asm-5.0.4-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-math3/3.5/commons-math3-3.5.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-api/1.7.15/slf4j-api-1.7.15.jar:/Users/huangqingshi/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/force-shading/1.7.2/force-shading-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-streaming-java_2.11/1.7.2/flink-streaming-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-runtime_2.11/1.7.2/flink-runtime_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-queryable-state-client-java_2.11/1.7.2/flink-queryable-state-client-java_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-hadoop-fs/1.7.2/flink-hadoop-fs-1.7.2.jar:/Users/huangqingshi/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-netty/4.1.24.Final-5.0/flink-shaded-netty-4.1.24.Final-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-jackson/2.7.9-5.0/flink-shaded-jackson-2.7.9-5.0.jar:/Users/huangqingshi/.m2/repository/commons-cli/commons-cli/1.3.1/commons-cli-1.3.1.jar:/Users/huangqingshi/.m2/repository/org/javassist/javassist/3.19.0-GA/javassist-3.19.0-GA.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/scala-library/2.11.12/scala-library-2.11.12.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.4.20/akka-actor_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/config/1.3.0/config-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-java8-compat_2.11/0.7.0/scala-java8-compat_2.11-0.7.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-stream_2.11/2.4.20/akka-stream_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/reactivestreams/reactive-streams/1.0.0/reactive-streams-1.0.0.jar:/Users/huangqingshi/.m2/repository/com/typesafe/ssl-config-core_2.11/0.2.1/ssl-config-core_2.11-0.2.1.jar:/Users/huangqingshi/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-protobuf_2.11/2.4.20/akka-protobuf_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.4.20/akka-slf4j_2.11-2.4.20.jar:/Users/huangqingshi/.m2/repository/org/clapper/grizzled-slf4j_2.11/1.3.2/grizzled-slf4j_2.11-1.3.2.jar:/Users/huangqingshi/.m2/repository/com/github/scopt/scopt_2.11/3.5.0/scopt_2.11-3.5.0.jar:/Users/huangqingshi/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill_2.11/0.7.6/chill_2.11-0.7.6.jar:/Users/huangqingshi/.m2/repository/com/twitter/chill-java/0.7.6/chill-java-0.7.6.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-clients_2.11/1.7.2/flink-clients_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-optimizer_2.11/1.7.2/flink-optimizer_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-shaded-guava/18.0-5.0/flink-shaded-guava-18.0-5.0.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.10_2.11/1.7.2/flink-connector-kafka-0.10_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-0.9_2.11/1.7.2/flink-connector-kafka-0.9_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/flink/flink-connector-kafka-base_2.11/1.7.2/flink-connector-kafka-base_2.11-1.7.2.jar:/Users/huangqingshi/.m2/repository/org/apache/kafka/kafka-clients/0.10.2.1/kafka-clients-0.10.2.1.jar:/Users/huangqingshi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/huangqingshi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.7/slf4j-log4j12-1.7.7.jar:/Users/huangqingshi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/huangqingshi/.m2/repository/com/alibaba/fastjson/1.2.62/fastjson-1.2.62.jar:/Users/huangqingshi/.m2/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:/Users/huangqingshi/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/Users/huangqingshi/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/Users/huangqingshi/.m2/repository/org/checkerframework/checker-qual/2.8.1/checker-qual-2.8.1.jar:/Users/huangqingshi/.m2/repository/com/google/errorprone/error_prone_annotations/2.3.2/error_prone_annotations-2.3.2.jar:/Users/huangqingshi/.m2/repository/com/google/j2objc/j2objc-annotations/1.3/j2objc-annotations-1.3.jar:/Users/huangqingshi/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.18/animal-sniffer-annotations-1.18.jar:/Users/huangqingshi/.m2/repository/redis/clients/jedis/3.1.0/jedis-3.1.0.jar:/Users/huangqingshi/.m2/repository/org/apache/commons/commons-pool2/2.6.2/commons-pool2-2.6.2.jar:/Users/huangqingshi/.m2/repository/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar:/Users/huangqingshi/.m2/repository/com/google/protobuf/protobuf-java/3.6.1/protobuf-java-3.6.1.jar:/Users/huangqingshi/.m2/repository/com/alibaba/druid/1.1.20/druid-1.1.20.jar:/Users/huangqingshi/.m2/repository/cglib/cglib/3.2.5/cglib-3.2.5.jar:/Users/huangqingshi/.m2/repository/org/ow2/asm/asm/5.2/asm-5.2.jar:/Users/huangqingshi/.m2/repository/org/apache/ant/ant/1.9.6/ant-1.9.6.jar:/Users/huangqingshi/.m2/repository/org/apache/ant/ant-launcher/1.9.6/ant-launcher-1.9.6.jar oom.DirectBufferOOM
Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:694)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at oom.DirectBufferOOM.main(DirectBufferOOM.java:18)
Process finished with exit code 1
Copy the codeIn this case, we use direct memory to cause overflow. At this time, we need to check whether the NIO used in the program and NIO, such as Netty, in the direct memory configuration.
0x06: A new error type was added after JDK1.6, which is reported if the heap memory is too small. This error is reported if less than 2% of the time is collected in 98% of the GC, that is, if there is a problem with the minimum and maximum memory. If the code is configured with a minimum and maximum heap of memory of 10m.
-Xmx10m -Xms10m package oom; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; /** * @Date 2020-07-18 */ public class GCOverheadOOM { public static void main(String[] args) { ExecutorService executor = Executors.newFixedThreadPool(5); for (int i = 0; i < Integer.MAX_VALUE; i++) { executor.execute(() -> { try { Thread.sleep(10000); } catch (InterruptedException e) { //do nothing } }); }}}Copy the codeThis creates a thread pool, and if the thread pool is executing and the core thread can’t handle it, it will put the data in LinkedBlockingQueue, which is the heap memory. In this case, we need to check whether the -xms-xmx min-max heap configuration is reasonable. Another dump shows the current memory to see if a lot of loops are being used or a lot of in-memory code is being used.
The above is the situation often encountered, need to analyze and deal with the emergence of different situations.
Reprinted from public account: Java Paradise
The original address: mp.weixin.qq.com/s/XJhtBYGMJ…