preface
While working with Node.js, MEMORY sometimes explodes at the drop of a switch, until I read about V8’s memory limits and garbage collection algorithms in Node.js, let me take a quick note.
V8 memory limits
For a 1.5GB heap, V8 takes 50 milliseconds to do a small garbage collection and 1s to do a non-incremental garbage collection. Moreover, garbage collection is fully paused, meaning that your application running process will pause and go to garbage collection (and Node is single-threaded), which is very unpleasant. Worse, on 64-bit systems, JS only uses about 1.4GB of memory, and on 32-bit systems, about 0.7GB. To open the memory limit, use the following two parameters
Node-max-old-space-size =1700 test.js # unit MB set the new generation memory node --max-new-space-size=1024 test.js # unit KB Set the new generation memoryCopy the codeV8 main garbage collection algorithm
What is the command for printing garbage collection logs in a specific scenario
Nodetest.js --trace_gc >gc.log # Output to log, otherwise to consoleCopy the codeOk, V8’s overall use of generational garbage collection
According to the survival of the object is divided into new generation and old generation.
The Scavenge avenge is split in two, with each part of the space being semispace, the one in use (From) and the one in idle (TO), and the garbage collection duplicates the living objects From TO, flipping the roles of the two.
Multiple copies of an object that is still alive (lifecycle) are transferred to the old generation space, which is called promotion.
Using a combination of Mark-sweep and Mark-Compact algorithms in older generations,
Mark-sweep marks all living objects and sweeps the unmarked (dead objects), resulting in the following image
Since this algorithm causes space fragmentation and wastes space (discontinuous memory areas), mark-Compact algorithm is introduced to move living objects to one end.
In the image below, the white squares are living objects, the dark ones are dead, and the light ones are moved space (original position).
Specific algorithms are described in reference books.
I see this information in the output garbage collection log
var a=[]; for(var i=0; i<1000000; i++){ a.push(new Array(1000)) var b=12; delete b; }Copy the codenode app.js --trace_gc >gc.log [4188:000001F722B22570] 37 ms: Avenge 3.8 (4.0) -> 3.7 (5.0) MB, 1.1/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 38 ms: Insane 3.8 (5.0) -> 3.6 (6.3) MB, 0.9/0.0 ms (average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 38 ms: Insane 4.6 (6.3) -> 4.5 (8.3) MB, 0.5/0.5 ms (average mu = 1.000, Current mu = 1.000) Allocation Failure [4188:000001F722B22570] 40 ms: Avenge 5.6 (8.3) -> 5.6 (9.0) MB, 1.0/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 42 ms: Avenge 6.5 (9.0) -> 6.5 (13.8) MB, 1.3/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 44 ms: Avenge 9.4 (14.8) -> 9.5 (14.8) MB, 1.5/0.0 ms (average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 47 ms: Insane 10.3 (14.8) -> 9.9 (14.8) MB, 5.5/0.0 ms (average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 52 ms: Avenge 17.0 (25.8) -> 17.7 (26.5) MB, 5.5/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 56 ms: Insane 17.7 (26.5) -> 16.7 (49.5) MB, 5.0/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 65 ms: Avenge 32.5 (49.5) -> 34.0 (51.0) MB, 5.5/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 71 ms: Avenge 34.0 (51.0) -> 32.0 (65.8) MB, 5.5/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 76 ms: Exploiture 47.7 (65.8) -> 49.2 (66.8) MB, 5.0/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 82 ms: Avenge 49.2 (66.8) -> 47.2 (80.8) MB, 5.5/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 87 ms: Avenge 62.9 (80.8) -> 64.4 (82.0) MB, 4.0/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 93 ms: Avenge 64.4 (82.0) -> 62.4 (96.0) MB, 5.3/0.0 ms (Average mu = 1.000, Current MU = 1.000) Allocation Failure [4188:000001F722B22570] 210 ms: Mark-sweep 134.6 (166.2) -> 133.7 (169.2) MB, 75.9/0.0 ms (+ 2.3ms in 9 steps since start of marking, Biggest step 0.7 ms, walltime since start of marking 80 ms) (Average mu = 1.000, Current MU = 1.000) Finalize Incremental marking via stack Guard GC in old space requisition [4188:000001F722B22570] 751 ms: Mark-sweep 554.1 (597.2) -> 553.3 (596.4) MB, 320.5/0.0 ms (+ 2.1 ms in 9 steps since start of marking, Biggest step 0.7 ms, walltime since start of marking 326 ms) (Average mu = 0.404, Current MU = 0.404) Finalize Incremental marking via stack Guard GC in old space requested [4188:000001F722B22570] 1944 ms: Mark-sweep 1305.2 (1361.8) -> 1303.8 (1360.3) MB, 766.3/0.0 ms (+ 2.3ms in 9 steps since start of marking, Biggest step 0.7ms, walltime since start of marking 774 ms) (Average mu = 0.371, Current MU = 0.356) Finalize Incremental marking via stack Guard GC in old space requested [4188:000001F722B22570] 3164 ms: Mark-sweep 1680.2 (1743.6) -> 1678.8 (1742.2) MB, 1010.3/0.0 ms (+ 2.0 ms in 9 steps since start of marking, Biggest step 0.7 ms, walltime since start of marking 1018 ms) (Average mu = 0.254, Current MU = 0.171) Finalize Incremental marking via stack Guard GC in old space requested [4188:000001F722B22570] 4538 ms: Mark-sweep (reduce) 1850.0 (1916.5) -> 1850.0 (1885.5) MB, 1254.3/0.0 ms (+ 3.3ms in 9 steps since start of marking, Biggest step 1.2 ms, walltime since start of marking 1265 ms) (Average mu = 0.158, Current MU = 0.085) Finalize Incremental marking via stack Guard GC in old space requested [4188:000001F722B22570] 5914 ms: Mark-sweep (reduce) 1949.0 (1986.2) -> 1949.0 (1986.2) MB, 1285.7/0.0 ms (+ 0.2ms in 3 steps since start of marking, Biggest step 0.1 ms, walltime since start of marking 1292 ms) (Average MCopy the codeYou can see that the garbage collection log does have the names Scavenge and Mark_sweep in it, and can be credited to specific garbage collection times.
Just start to learn Node, might as well look at the garbage collection log, see which section of the code garbage collection time is too long, how to avoid these problems, write high-performance code, I think, this is the current start on the big project as soon as the online bug layer upon layer of error the best solution.
This concludes V8’s garbage collection system and will be covered later when we learn more about it. For details, I recommend reading Node.js in Depth.