A Memory Leak is a condition in which a program fails to release Memory that is no longer in use because of an oversight or error. If the location of the memory leak is critical, it is possible to hold more and more unused memory as the processing progresses, causing the server to slow down and, in severe cases, reaching a certain memory limit (it could be the upper limit of the process, such as v8’s limit; It can also be the maximum amount of memory the system can provide) that causes the application to crash. In traditional C/C++, there are memory leaks caused by wild Pointers, unreleased objects, etc. However, languages that use virtual machines, such as Java and JavaScript, use the Garbage Collection (GC) mechanism to automatically release memory, which greatly liberates programmers’ energy and eliminates the fear of releasing memory like traditional languages.

However, even with the automatic release of GC, this does not mean that memory leaks do not exist. Memory leaks are still a problem that developers can’t get around, and today we’ll look at how to analyze memory leaks in Node.js.

Node.js uses V8 as the execution engine for JavaScript, so talking about NODE.js GC is like talking about V8 GC. Whether an object’s memory is freed in V8 depends on whether there is room in the program to hold a reference to the changed object.

In V8, references to root objects (window in browser, global in Node.js) are sorted through each GC, and V8 marks them as reachable if they can be accessed from root’s reference chain, and unreachable if they can’t. Objects marked as unreachable (that is, objects without references) are reclaimed by V8. For more details, see Alinode’s Uncovering V8 GC.

As you can see from the above points, the cause of memory leaks in Node.js is that objects that should have been cleaned up, referenced by reachable objects, are not cleaned up properly and remain in memory.

Several cases of memory leakage

a = 10; // No object is declared.Copy the code
global.b = 11; // Global variable referencesCopy the code

For this simple reason, global variables hang directly on the root object and are not cleaned up.

Second, the closure

function out() { const bigData = new Buffer(100); inner = function () { void bigData; }}Copy the code

Closures refer to variables in parent functions, and if the closure is not released, it can cause a memory leak. In the example above, the inner hangs directly on the root, causing a memory leak (bigData will not be freed).

Note that the example here is simply hanging a reference to a global object, while the actual business situation may be caused by hanging on an object that can be traced back to root.

3. Event monitoring

Node.js event listeners can also leak memory. For example, if you listen to the same event repeatedly and forget to remove it (removeListener), memory leaks will occur. This can easily happen when adding events to a reusable object, so event repeat listeners may receive warnings like:

(Node :2752) Warning: Possible EventEmitter Memory leak detected. 11 Haha listeners Added. The Use of emitter. setMaxListeners() to increase limit

For example, node.js may cause memory leaks if the keepAlive value of the Agent is true. When Agent keepAlive is true, the socket used before will be reused. If you add event listening to the socket and forget to clear it, the reuse of the socket will lead to repeated event listening and memory leakage.

The principle is the same as the previous add event listener forgot to clear. When using HTTP modules in Node.js, there is no problem with reuse through keepAlive, which can cause memory leaks. So, you need to be aware of the life cycle of the object to which you add events to listen, and be careful to remove them yourself.

For an example of this issue, see Github’s Issues section.

Other reasons

There are other situations that can cause memory leaks, such as caching. When using a cache, you need to be aware of how many objects are in the cache. If there are too many objects in the cache, you need to limit the maximum number of cached objects. In addition, cpu-heavy code will also lead to memory leaks. When the server is running, if there is high CPU synchronous code, node.js is a single thread, so it can not handle processing requests, and request accumulation leads to high memory usage.

Locate memory leaks. 1. Reproduce the memory leaks

To locate a memory leak, there are two common scenarios:

For memory leaks that can be reproduced with normal use, this is a simple case to troubleshoot by simulating in a test environment.

Occasional memory leaks are usually associated with specific inputs. It’s a time-consuming process to reproduce this input steadily. If this particular input cannot be located through the code log, it is recommended to print a memory snapshot in production. Note that printing a memory snapshot consumes a lot of CPU and may affect online services.

You are advised to use Heapdump to save memory snapshots and devtool to view memory snapshots. When heapdump is used to save a snapshot in memory, only objects in the Node.js environment are left undisturbed (if node-inspector is used, front-end variables will interfere with the snapshot).

NPM install heapdump-target = node.js to install heapdump on node.js. 2. Print a memory snapshot

Add Heapdump to your code and use heapdump.writeSnapshot to print memory snapshots. To reduce interference with normal variables, the active memory release GC () function can be called before printing the memory snapshot (which can be turned on with the — expose-GC parameter at startup).

const heapdump = require('heapdump');
​
const save = function () {
  gc();
  heapdump.writeSnapshot('./' + Date.now() + '.heapsnapshot');
}
Copy the code

When printing code on the line, it is recommended to print snapshots based on memory growth. Heapdump can use kill to signal a program to print a memory snapshot (only available on * NIx systems).

Kill -usr2 You are advised to print three memory snapshots: one before the memory leak, one after a few tests, and one after multiple tests.

The first memory snapshot is used as a comparison to see which objects grow after the test. In cases where memory leaks are not obvious, they can be compared to memory snapshots taken after a large number of tests so that they are easier to locate.

Third, compare the memory snapshot to find the leak location

Through the memory snapshot to find the increasing number of objects, find the object is added by whom to reference, find the problem code, after the correction, specific problem specific analysis, here through our work to explain the situation.

const {EventEmitter} = require('events');
const heapdump = require('heapdump');
​
global.test = new EventEmitter();
heapdump.writeSnapshot('./' + Date.now() + '.heapsnapshot');
​
function run3() {
  const innerData = new Buffer(100);
  const outClosure3 = function () {
    void innerData;
  };
  test.on('error', () => {
    console.log('error');
  });
  outClosure3();
}
​
for(let i = 0; i < 10; i++) {
  run3();
}
gc();
​
heapdump.writeSnapshot('./' + Date.now() + '.heapsnapshot');
Copy the code

Here is a minimal reproduction of the error code.

First run the code with Node — expose-GC index.js to get two memory snapshots, then open DevTool and click Profile to load the memory snapshots. When contrast is turned on, Delta shows how the object has changed. If the object Delta keeps growing, there is a good chance of a memory leak.

You can see three areas where objects grow significantly: closures, context, and Buffer objects. Click to see how the object is referenced:

In fact, these three object growth is caused by a problem. The closure references the innerData object in the error listening event in the test object, causing the buffer to not be cleaned and resulting in a memory leak.

The innerData object is referenced in the closure of the error listener event, but the innerData object is not referenced in the closure. Comparing snapshots to find problems depends on how familiar you are with the code and how discerning you are.

How to avoid memory leaks The examples in this article are very clear to see memory leaks, but in the work, the code mixed with business will not be very clear to see memory leaks, and still rely on tools to locate memory leaks. Here are some other ways to avoid memory leaks.

ESLint checks code for unexpected global variables.

When using a closure, you need to know what objects the closure contains and when objects referencing the closure are cleared. It is best to avoid writing out complex closures, because memory leaks caused by complex closures are hard to see without printing a memory snapshot.

When binding events, be sure to clear them at the appropriate time. When writing a class, it is recommended to use the init function to bind the class’s event listeners and request resources, and then destroy to release the events and resources.

As a bonus note, the following closure summary came out after a lot of testing.

class Test{}; global.test = new Test() function run5(bigData) { const innerData = new Buffer(100); // Referenced by the closure, create a context: context1. // context1 references bigData, innerData. // Closure for function run5() // The run5 function has no context, so context1 has no previous. // The newly created function in run5 will bind context1. Test.outclosure5 = function () {// This function closure refers to context1. void bigData; const closureData = new Buffer(100); // Used by closures, create context: context2. // outClosure5 has context1, previous points to context1. // The new function in outClosure5 will bind context2. Test.innerclosure5 = function () {// This function closure context points to context2. void innerData; } test.innerclosure5_1 = function () {// This function closure context points to context2. void closureData; }}; test.outClosure5_1 = function () { } test.outClosure5(); } run5(new Buffer(1000));Copy the code

V8 will generate a context internal object to implement the closure. Here are the rules for generating context in V8.

V8 creates a context2 at the closure reference variable declaration. If the function of the closed variable has context1, the previous of the created context2 points to context1. A function created within the function referenced by the closure will bind context2.

Since this is related to the V8 version, only V6.2.2 and V6.10.1 and V7.7.1 are tested here, and the same is true. You can learn more about this REPO if you want to practice testing.

Note: This article is published by Ele. me big front-end Node group, written by @Wang Dashuai of our group, and organized by @Lellansin.