This article was originally published on the public account CoyPan as expected

This is a translation, partly abridged. Deepu. Tech/Memory-Mana…

Visualizing Memory Management in V8 Engine (JavaScript, NodeJS, Deno, WebAssembly)

In this chapter, we’ll cover memory management for the V8 engines for ECMAScript and WebAssembly, which are used for NodeJS, Deno&Electron, and so on, And web browsers like Chrome, Chromium, Brave, Opera and Microsoft Edge. Because JavaScript is an interpreted language, it requires an engine to interpret and execute the code. The V8 engine interprets JavaScript and compiles it to machine code. V8 is written in C++ and can be embedded in any C++ application.

First, let’s look at the MEMORY structure of the V8 engine. Because JavaScript is a single-threaded language, V8 uses one process for each JavaScript context. If you use a service worker, V8 will start a new process for each service worker. In the V8 process, a running program is always represented by a number of allocated memory, called a Resident Set. The following different sections can be further divided:

This is somewhat similar to the JVM we mentioned in our last article. Let’s take a look at what each part does:

Heap memory

This is where V8 stores objects and dynamic data. This is the largest area in memory and is where garbage collection (GC) occurs. The entire heap memory is not garbage collected; only the Old and New Spaces are garbage collected. Heap memory can be further divided into the following sections:

  1. New Space

The new space, or “new generation”, is where new objects are stored, and most of them have a very short declaration cycle. This space is small, with two half-spaces, similar to S0 and S1 in the JVM. This space is managed by **Scavenger(Minor GC)**, which will be covered later. The size of the new generation space can be controlled by two V8 flags –min_semi_space_size(the initial value) and –max_semi_space_size(the maximum value).

  1. Old Space

Old space (or “old generation”) stores data that survived two Minor GC’s in the new generation space. This space is managed by **Major GC(Mark-sweep & Mark-Compact) “**, which is covered later. The size of the old generation space can be controlled by two V8 flags –initial_old_space_size(the initial value) and –max_old_space_size(the maximum value). The space is divided into two parts:

  • Old pointer space: Contains surviving objects that contain Pointers to other objects.
  • Old data space: Contains objects that only hold data (there are no Pointers to other objects). Strings, boxed numbers, and unboxed double arrays that survive two rounds of Minor GC in the new generation space are moved to the old data space.
  1. Large Object Space

This is where objects larger than other space size limits are stored. Each object has its own area of memory. Large objects are not garbage collected.

  1. Code-space

This is where the just-in-time (JIT) compiler stores blocks of compiled code. This is the only space with executable memory (although code may be allocated in “large object space,” they are also executable).

  1. Cell space, property Cell space, and Map space

These Spaces contain the Cell, PropertyCell, and Map respectively. Each of these Spaces contains objects of the same size, and there are some restrictions on the type of object they point to, which simplifies collection.

Each space consists of a set of pages. Pages are contiguous chunks of memory allocated from the operating system using MMAP. Each page size is 1MB, but large objects have more space.

The Stack (Stack)

This is the area of stack memory, one for each V8 process. Static data is stored here, including method/function frames, primitive values, and Pointers to objects. The stack memory limit can be set using the –stack_size V8 flag.

V8 memory usage (stack vs. heap)

Now that we know how memory is organized, let’s look at how to use the most important parts of it when executing a program.

Let’s use the following JavaScript program. The code is not optimized for correctness, so it ignores unnecessary intermediate variables, etc., and focuses on visualizing stack and heap memory usage.

class Employee {
    constructor(name, salary, sales) {
        this.name = name;
        this.salary = salary;
        this.sales = sales; }}const BONUS_PERCENTAGE = 10;

function getBonusPercentage(salary) {
    const percentage = (salary * BONUS_PERCENTAGE) / 100;
    return percentage;
}

function findEmployeeBonus(salary, noOfSales) {
    const bonusPercentage = getBonusPercentage(salary);
    const bonus = bonusPercentage * noOfSales;
    return bonus;
}

let john = new Employee("John".5000.5);
john.bonus = findEmployeeBonus(john.salary, john.sales);
console.log(john.bonus);
Copy the code

The following slide shows how stack and heap memory are used during the execution of the above code.

As you can see:

  1. The Global scope is stored in the Global frame on the stack.
  2. Each function call is added to stack memory as a frame block.
  3. All local variables (including parameters and return values) are stored in the function box block of the stack.
  4. All primitive types like INT&String are stored directly on the stack. The same applies to global scopes.
  5. The function called by the current function is pushed to the top of the stack.
  6. When the function returns, its frame block is removed.
  7. Once the main process completes, objects on the heap no longer have Pointers from the stack and become isolated objects.
  8. Unless explicitly copied, all object references in other objects are done using reference Pointers.

As you can see, the stack is automatically managed by the operating system, not V8. Therefore, we don’t have to worry too much about stacks. The heap, on the other hand, is not automatically managed by the operating system, and because the heap is the largest memory space and holds dynamic data, it can grow exponentially over time, causing our program to run out of memory. It also becomes fragmented over time, slowing down applications. That’s why recycling is needed.

Distinguishing between Pointers and data on the heap is important for garbage collection, and V8 uses the “marker pointer” approach to achieve this. In this approach, it reserves a bit at the end of each word to indicate whether it is a pointer or data. This approach requires limited compiler support, but is simple to implement and quite efficient.

V8 Memory Management – Garbage Collection (GC)

Now that we know how V8 allocates memory, let’s look at how it automatically manages heap memory, which is important for application performance. When a program tries to allocate more memory on the heap than is freely available (depending on the V8 flag set), we encounter out-of-memory errors. Mismanaged heaps can also lead to memory leaks.

V8 manages heap memory through garbage collection. Simply put, it frees up memory used by isolated objects (that is, objects that are no longer referenced directly or indirectly from the stack (through a reference in another object) to make room for the creation of new objects.

Orinoco is the code name of the V8 GC project used to free the main thread using parallel, incremental, and concurrent garbage collection techniques.

The garbage collector in V8 is responsible for reclaiming unused memory for reuse by V8 processes.

The V8 garbage collector is generational (objects in the heap are grouped by age and purged at different stages). V8 has two phases and three different garbage collection algorithms:

Minor GC (Scavenger)

This type of GC keeps the Cenozoic space compact and clean. Objects are allocated into fairly small Spaces (between 1 and 8MB, depending on the behavior heuristic). The cost of allocating new generation space is low: there is an allocation pointer that increments every time we want to reserve space for new objects. When the allocation pointer reaches the end of the Cenozoic space, the subminor GC is triggered. This process, called Scavenger, implements the “Cheney algorithm.” Minor GC often appears and uses parallel worker threads, and is very fast.

Let’s take a look at the Minor GC process:

Cenozoic space is divided into two half Spaces of equal size: from-space and to-space. Most allocations are made in to-space (except for certain types of objects, such as executable code that is always allocated in old-generation space). When the to-space fills, the Minor GC is triggered. The completion process is as follows:

  1. When we start, assume that there are already objects in the to-space.

  2. The process creates a new object.

  3. V8 tried to fetch the required memory from to-space, but there was no space available to hold our objects, so V8 triggered the Minor GC.

  4. The Minor GC swaps to-space and FROm-space, all objects are now in from-space, and to space is empty.

  5. The Minor GC recursively traverses the object graph in from-space, starting from the stack pointer (the GC root), looking for used or active objects (used memory). These objects will be moved to the pages of the to-space. Any objects referenced by these objects are also moved to the page in to-space, and their Pointers are updated. Repeat this until all objects in from-space have been scanned once. Eventually, the to-space is automatically compressed to reduce fragmentation.

  6. The Minor GC now empties from-space because any remaining objects here are garbage.

  7. The new object is allocated to the memory space of the to-space.

  8. Let’s assume that over time, there are more objects in the to-space.

  9. The application creates a new object.

  10. V8 tried to fetch the required memory from to-space, but there was no space available to hold our objects, so V8 triggered a second Minor GC.

  11. Repeat the process and move any live objects that survive the second Minor GC to the old generation space. The survivors of the first Minor GC are moved to to-space and the remaining garbage is removed from from-space.

  12. The new object is allocated to the memory space of the to-space.

We saw how the Minor GC reclaimed space from the new generation memory space and kept it compact. This process stops all other operations, but it is fast and efficient, and most of the time trivial. Since this process does not scan objects in the old generation space for any references in the new generation space, it uses registers for all Pointers from the old generation space to the new generation space. This will be recorded into the storage buffer by a process called Write Barriers.

Major GC

This type of GC keeps the old space compact and clean. This operation is triggered when V8 determines, based on the constraints of dynamic computation, that there is not enough old generation space, because it was filled from a Minor GC cycle.

The Scavenger algorithm is well suited for small data volumes, but is not practical for large old-generation Spaces because of its memory overhead, so the main GC is done using the Mark-sweep-Compact algorithm. It uses a three-color (white, grey and black) marking system. Therefore, the Major GC is a three-step process, with the third step being performed according to segmentation heuristic.

  • Marking: The first step is common to both algorithms, where the garbage collector identifies which objects are being used and which are not. Objects that are in use or accessible recursively from the GC root (stack pointer) are marked as active. Technically, this is a depth-first search of the heap and can be viewed as a directed graph.
  • Cleanup: The garbage collector walks through the heap and records the memory address of any object not marked as active. These Spaces are now marked as free in the free list and can be used to store other objects.
  • Compress: After cleaning, move all remaining objects together if necessary. This reduces fragmentation and improves the performance of allocating memory to newer objects.

This type of GC is also called stop-the-world GC because they introduce pause times during GC execution. To avoid this V8, the following techniques are used:

  • Incremental GC: GC is done in multiple incremental steps instead of one incremental step.
  • Concurrent tagging: Tagging is done concurrently using multiple helper threads without affecting the main JavaScript thread. Write Barriers are used to track new references between objects created by JavaScript when the helper program concurs tags.
  • Concurrent scanning/compression: Scanning and compression are done simultaneously in the helper thread without affecting the main JavaScript thread.
  • Deferred cleanup: Deferred cleanup, including deferred removal of garbage from pages, until memory is required.

Let’s take a look at the major GC process:

  1. Let’s assume that many Minor GC cycles have passed, the old space is almost full, and V8 decides to trigger a Major GC
  2. The Major GC traverses the object graph recursively, starting with the stack pointer, to mark objects that are used as active (used memory) and remaining objects as garbage (isolated) in the old generation space. This is done using multiple concurrent helper threads, each following a pointer. This does not affect the main JS thread.
  3. GC performs the tag termination step using the main thread when the concurrent tag completes or the memory limit is reached. This will introduce a small pause time.
  4. The Major GC now uses concurrent scanning threads to mark the memory of all isolated objects as free. Parallel compression tasks are also triggered to move related memory blocks to the same page to avoid fragmentation. Pointers are updated during these steps.

conclusion

This article gives you an overview of V8 memory structure and memory management. Not everything is covered here, but there are many more advanced concepts that you can learn from v8. Dev. But for most JS/WebAssembly developers, this level of information is enough, and I hope it helps you write better code. With these considerations in mind, for higher-performing applications, keeping this in mind can help you avoid the next memory leak problem you might encounter.