Any used based on Java enterprise backend application software developers will be encountered this kind of poor, strange error, the error from the user or test engineer: Java. Lang. OutOfMemoryError: Java heap space.

To figure this out, we must return to the computer science foundations of algorithmic complexity, especially “spatial” complexity. If we recall, every application has a worst-case characteristic. Specifically, in terms of storage dimensions, more storage than recommended will be allocated to the application, which is an unpredictable but acute problem. This leads to an overuse of heap memory, hence the “out of memory” situation.

The worst part of this particular situation is that the application cannot be fixed and will crash. Any attempt to restart the application – even using maximum memory (-xMX option) – is not a long-term solution. Without understanding what causes heap usage to swell or bulge, memory usage stability (i.e., application stability) cannot be guaranteed. So, what’s a more effective way to understand programming problems with memory? Understanding the heap and distribution of your application’s memory will answer this question when memory runs out.

In this context, we will focus on the following aspects:

  • When memory runs out, a heap dump is retrieved from the Java process.

  • Understand the types of memory problems your application is experiencing.

  • Using a heap analyzer, you can use Eclipse MAT, an excellent open source project, to analyze memory overflow problems.

Configure the application in preparation for heap analysis

Any nondeterministic, on-again, off-again problem like memory overflow is a challenge for post-mortem analysis. Therefore, the best way to handle a memory overflow is to have the JVM virtual machine dump a heap file of the MEMORY state of the JVM virtual machine.

The Sun HotSpot JVM has a way of instructing the JVM to dump the heap state of a memory overflow into a file. The standard format is.hprof. So, in order to realize the operation to the JVM startup add XX: + HeapDumpOnOutOfMemoryError. Adding this option to production systems is also necessary because memory overruns can occur over a long period of time.

If the heap dump. Hprof file must be written to a specific file system location, then add the directory path to XX:HeapDumpPath. Just make sure that the application always has write permission for the specified directory path.

Cause analysis,

101: Understand the nature of memory overflow errors

When trying to evaluate and understand a memory overflow error, the first thing you should do is look at the memory growth characteristics. Make a probability assessment based on the situation:

  • Spike: This type of memory overflow can be intense for certain types of loading. When the JVM allocates memory to 20 users, the application runs fine. However, if you hit the 100th user, you might hit a memory peak, causing a memory overflow. There are two possible ways to solve this problem.

  • Leak: Memory usage has increased over time due to some programming issues.

A health chart with a healthy garbage collection mechanism

     

Chart that leaks over time after a period of health

          

Memory chart that causes memory usage to bulge, causing memory overflow

Once we understand the nature of the memory problem that caused the surge in usage, based on what we infer from our analysis, the following methods may be used to avoid memory overflow errors.

Solving memory problems

  1. Fix code causing memory overflow: A bug had to be fixed because the application incrementally added an object over a period of time without clearing its references (object references from the running application). For example, this error might be the insertion of a hash table in which the business objects grow, but the business logic and transaction do not remove these objects after completion.

  2. Increase the memory maximum as a fix. Once you understand the running memory characteristics and the heap, you may have to increase the maximum heap memory allocated to avoid memory overruns because the recommended maximum memory value is not sufficient for the stability of your application. Therefore, the application may have to update the Java-XMX flag information to a higher value based on the heap analyzer’s evaluation before running it.

Heap analysis

Let’s take a closer look at how to analyze heap dumps using a heap analysis tool. In this example, the Eclipse Foundation’s open source tool MAT will be used.

Use MAT for heap analysis

It’s time to dig deeper. We’ll walk through a series of steps to help explore different representations and views in MAT to get an example of a heap overflow and think about analysis.

1. Open the.hprof heap file generated when the memory overflow error occurred. Make sure to copy the dump files to a special folder, as MAT creates many index files: File -> Open

2. Open the dump file. There are options to report suspected memory leaks and components. Select run leak suspicion report.

3. After the leak suspect list is opened, the pie chart in the preview window will show the distribution of reserved memory on the basis of each object. It shows the largest objects in memory (objects with the highest reserved memory — accumulated memory and referenced objects).

4. The pie chart above shows the three problem suspects by aggregating the objects with the highest memory references (own memory and total memory).

Let’s look at each case and evaluate whether it is the root cause of the memory overflow error.

Suspects 1

The 454,570 instances of java.lang.ref.Finalizer loaded by < System Class Loader > occupied 790,205,576 bytes (47.96%).

This tells us that there are 454,570 instances of JVM Finalizers that account for nearly 50 percent of the allocated application memory.

Assuming readers know what Java Finalizer is for, what does the above information tell us?

Introduction to reading: stackoverflow.com/questions/2…

In essence, developers write some custom finalizers to free an instance’s resources. These instances collected by finalizers are outside the scope of the JVM’s garbage collection algorithm using separate queues. In fact, this path is longer than the garbage collection mechanism’s cleanup path. So now we should try to figure out what exactly do these terminators end?

Also may be 2 suspects, accounted for 20% of the sun. Security. SSL. SSLSocketImpl. Can we confirm whether these are the instances to be terminated by the finalizer?

Suspects 2

Now, let’s open the Dominator view below the tools button at the top of the MAT. We will see all listed class instances parsed by MAT to show valid heap storage.

Next, the Dominator view, we try to understand Java. Lang. Finalizers and sun. Security. SSL. The relationship between the SSLSocketImpl. We right click on the sun. Security. SSL. SSLSocketImpl this column, open the GC Roots – > exclude soft/weak references.

MAT will now begin to graph memory to show the path of GC Root and its corresponding instance references. This will be displayed on another page with the following reference:

As shown in the above reference chain, the instance SSLSocketImpl comes from java.lang.ref.finalizer, and the entire instance of SSLSocketImpl takes approximately 88K. We also notice that the Finalizer chain is a pin list data structure that points to the next instance.

Corollary: At this point, we have a definite feeling that Java Finalizer is trying to collect SSLSocketImpl objects. To explain why there is so much information that has not been collected, I examine the code.

Check the code

The code check needs to see if the socket is closed. In this case, it shows all the I/ O-related streams that need to be shut down correctly. At one point, we suspect that the JVM is the culprit. In fact, there is a BUG in the code in the Open JDK 6.0.xx’s GC (garbage collector).

I hope this article gives you a pattern for analyzing whether errors in Java applications are caused by heap storage or internal problems. I hope you enjoy using heap analysis!

Further reading

Shallow heap vs. Retained heap

Shallow heap is the memory consumed by an object. Depending on the case, an object needs to be 32 or 64 bits (depending on its operating system architecture), 4 bytes for integers, 8 bytes for longs, and so on. Depending on the heap dump format, its memory size (for example, aligned to 8) may be appropriate to better shape the virtual machine’s true consumption.

The reserved set of X is the set of objects that will be removed when X is garbage collected.

The reserved heap of X is the sum of the shallow heaps of all objects in the reserved set of X, that is, the memory reserved by X.

In general, the shallow heap of an object is its size in the heap. The reserved size of the same object is the amount of heap memory when the object is garbage collected.

The reserved set of major sets of objects, such as all objects of a particular class, or all objects of all classes loaded by a particular classloader, or just arbitrary objects, is the set of objects to be released if all objects of those major sets become inaccessible. The reserved set includes these objects and other objects that can be obtained only from these objects. The size of the reserved set is the size of the heap of all objects contained in the reserved set.