Most web sites do not require a lot of computation, and the program’s time is focused on disk I/O and network I/O

SSDS are fast reads, but not on the order of magnitude as fast as the CPU can process instructions, and the round-trip time of a packet across the network is slower:

The average latency of a packet back and forth is 320ms(I have a slow Internet connection, but ping a domestic website is faster), which should be fine for a normal CPU executing tens of millions of cycles

This is where asynchronous IO comes into play. For example, with multithreading, if you use Java to read a file, it is a blocking operation that does nothing while waiting for the data to return, so you start a new thread to handle the file read and notify the main thread when the read is finished.

That makes sense, but it’s hard to write the code. What about node.js V8 that cannot open a thread?

What is a Node.js process

We can silently answer the following 9 questions, are they clear?

1.1 asynchronous I/o

Asynchronous IO refers to the IO (data in and out) capability provided by the operating system, such as keyboard input, corresponding to the display will have a special data output interface, this is our life visible IO capability; This interface then goes down to the operating system level, where the operating system provides many capabilities, such as disk read and write, DNS query, database connection, network request processing, and so on.

At the level of different operating systems, the performance is inconsistent. Some are asynchronous and non-blocking; Some are synchronous blocking, in any case, we can think of it as the data interaction between the upper application and the lower system; The upper layer depends on the lower layer, but in turn, the upper layer can modify the capabilities provided by the lower layer; If the operation is asynchronous and non-blocking, then this is the asynchronous non-blocking asynchronous I/O model; If it is synchronous blocking, then it is synchronous IO model;

Koa is a top-level Web services framework, all by JS implementation, it has the interaction between operating systems, all through nodeJS to achieve; For example, nodeJS readFile is an asynchronous non-blocking interface, and readFileSync is a synchronous blocking interface. The above three questions are basically answered here;

1.2 Event Loop

Event loops are where Node.js performs non-blocking I/O operations. Although JavaScript is single-threaded, since most kernels are multi-threaded, Node.js loads operations into the system kernel as much as possible. So they can handle multiple operations that are performed in the background. When one of the operations is complete, the kernel tells Node.js so that Node.js can add the corresponding callback to the polling queue for final execution.

1.3 summarize

Nodejs is single-threaded and is based on an event-driven, non-blocking IO programming model. This allows us to continue executing code without waiting for the result of the asynchronous operation to return. When an asynchronous event is triggered, the main thread is notified, and the main thread performs a callback to the event.

2. Nodejs architecture analysis

When talking about Nodejs architecture, we first need to look at the relationship and role of Nodejs with V8 and libUV:

  • V8: the engine that executes JS. Translation js. including the familiar compilation optimization, garbage collection and so on.
  • libUV:provideasync I/OTo provide a message loop. Is an abstraction layer of the OPERATING system API layer.

So how does Nodejs organize them?

2.1 the Application Code (JS)

Framework code and user code — the application code we write, the NPM package, the JS module built into NodeJS, etc. — we spend most of our daily work writing this layer of code.

2.2 the binding code

Binding code or third-party plug-in (JS or C/C++ code) glue code.

Code that lets JS call C/C++. You can think of it as a bridge, with js at one end and C/C++ at the other, through which JS can call C/C++. In NodeJS, the main role of glue code is to expose nodeJS to the UNDERLYING C/C++ libraries. The three party plug-in is our own C/C++ library, and we need to implement the glue code to bridge JS and C/C++.

Nodejs passes JS to V8 through a C++ Binding, V8 parses it and sends it to libUV to launch asnyc I/O, and waits for the message loop to schedule.

2.3 the underlying library

Nodejs dependencies include well-known V8 and Libuv.

  • V8: As we all know, it’s a set of efficient javascript runtimes developed by Google, and it’s a big reason nodeJS can execute JS code so efficiently.
  • Libuv: Libuv is a set of asynchronous functions implemented in C language, nodeJS efficient asynchronous programming model is largely due to the implementation of Libuv, and libuv is the focus of today’s analysis.

There are other dependency libraries

  • Http-parser: Parses HTTP responses
  • Openssl: encryption and decryption
  • C-ares: DNS resolution
  • NPM: Nodejs package manager

3. Libuv architecture

As we know, libuv is the core of NodeJS ‘asynchronous mechanism. Libuv acts as a bridge between NodeJS and asynchronous tasks such as files, networks, and so on. Here is an overview of Libuv:

This is a diagram of the libuv website. It is clear that nodeJS network I/O, file I/O, DNS operations, and some user code are all working in Libuv. Since we’re talking about asynchrony, let’s first summarize the asynchrony events in NodeJS:

3.1 Non-i /O operations:

  • Timer (setTimeout, setInterval)
  • Microtask (promise)
  • process.nextTick
  • setImmediate
  • DNS.lookup

3.2 I/O operations:

  • Network I/O

For network I/O, different platforms have different implementation mechanisms. Linux is epoll model, Unix-like Kquene, Windows is efficient IOCP completion port, SunOs is event ports. Libuv encapsulates these network I/O models.

  • File I/O and DNS operations

Libuv also maintains an internal thread pool of four threads by default, which are responsible for file I/O operations, DNS operations, and user asynchronous code. When the JS layer passes libuv an operation task, libuv queues the task. Then there are two cases:

When all threads in the thread pool are occupied, tasks in the queue are queued for idle threads.

2. When there are available threads in the thread pool, the thread is removed from the queue and executed. After the execution, the thread is returned to the thread pool and waits for the next task. It also notifies event-loop with an event, and event-loop executes the callback function registered for the event after receiving the event.

Of course, if four threads are not enough, you can set the UV_THREADPOOL_SIZE environment variable when nodejs starts. Libuv limits the number of threads to 128 for system performance.

4. Nodejs threading model

The node.js startup process can be divided into the following steps:

Initialize nodeJS by calling platformInit.

2, call performance_node_start method, nodeJS performance statistics.

3. Judge openSSL Settings.

Initialize the libuv thread pool by calling v8_platform.Initialize.

5. Call V8::Initialize to Initialize the V8 environment.

Create a nodejs run instance.

7. Start the instance created in the previous step.

8, start to execute JS file, synchronization code execution is completed, into the event loop.

9, when there is no event to listen to, destroy nodeJS instance, program execution is finished.

This is how nodejs executes a JS file. Let’s focus on the eighth step, the event loop.

Nodejs is completely single-threaded. From the start of the process, our js file (main.js in the figure below) is loaded from the main thread and we enter the message loop. Visible for THE JS program, the complete run in a single thread.

This does not mean that the Node process has only one thread. Node.js Event Loop Workflow & Lifecycle in Low level

4.1 Details the message loop

Take a look at the message loop in JS:

  • timers: Executes the expired callback in setTimeout() and setInterval().
  • I/O callbacks: A few I/ OCallbacks from the previous cycle will be delayed to this phase of this cycle
  • idle, prepare: For internal use only
  • poll: The most important phase in which I/O callbacks are executed and blocked under appropriate conditions
  • check: Perform setImmediate’s callback
  • close callbacks: Executes the callback for the close event, such as socket.on(“close”,func)

Nodejs subdivides the message loop into six phases (officially called phases), each of which has a queue-like structure that stores the callbacks that need to be processed in that Phase. Let’s take a look at the role of these six phases. The core code of these six phases is as follows:

int uv_run(uv_loop_t* loop, uv_run_mode mode) {
  int timeout;
  int r;
  int ran_pending;
// Determine if the event loop is alive.
  r = uv__loop_alive(loop);
  // If not, update the timestamp
  if(! r) uv__update_time(loop);// If the event loop is alive and the event loop is not stopped.
  while(r ! =0 && loop->stop_flag == 0) {
    // Update the current timestamp
    uv__update_time(loop);
    // Execute the Timers queue
    uv__run_timers(loop);
    // Execution is delayed to the I/O callback of this loop because the last loop did not finish.
    ran_pending = uv__run_pending(loop); 
    // Internal call, user does not care, ignore
    uv__run_idle(loop); 
    // Internal call, user does not care, ignore
    uv__run_prepare(loop); 
    
    timeout = 0; 
    if((mode == UV_RUN_ONCE && ! ran_pending) || mode == UV_RUN_DEFAULT)// Calculate the time difference between the arrival of the next timer.
      timeout = uv_backend_timeout(loop);
   If yes, I/O events will be executed. If no, I/O events will be blocked until the timeout time is exceeded.
    uv__io_poll(loop, timeout);
    // Enter the check phase and perform the setImmediate callback.
    uv__run_check(loop);
    // Perform the close phase, which mainly executes the ** close ** event
    uv__run_closing_handles(loop);

    if (mode == UV_RUN_ONCE) {
      
      // Update the current timestamp
      uv__update_time(loop);
      // Run the timers callback again.
      uv__run_timers(loop);
    }
    // Determine whether the current event loop is alive.
    r = uv__loop_alive(loop); 
    if (mode == UV_RUN_ONCE || mode == UV_RUN_NOWAIT)
      break;
  }

  /* The if statement lets gcc compile it to a conditional store. Avoids * dirtying a cache line. */
  if(loop->stop_flag ! =0)
    loop->stop_flag = 0;

  return r;
}

Copy the code

4.2 the Timer Phase

This is the first phase of the message loop, with a for loop handling all setTimeout and setInterval callbacks. The core code is as follows:

void uv__run_timers(uv_loop_t* loop) {
  struct heap_node* heap_node;
  uv_timer_t* handle;

  for (;;) {
  // Fetch the handle of the timer with the closest timeout time in the timer heap
    heap_node = heap_min((struct heap*) &loop->timer_heap);
    if (heap_node == NULL)
      break;
    
    handle = container_of(heap_node, uv_timer_t, heap_node);
    // Check whether the timeout period of the latest timer handle is greater than the current time. If the timeout period is greater than the current time, it indicates that the timer handle has not timed out and the loop is broken out.
    if (handle->timeout > loop->time)
      break;
    // Stop the nearest timer handle
    uv_timer_stop(handle);
    // Check whether the timer handle type is repeat. If yes, create a timer handle again.
    uv_timer_again(handle);
    // Execute the callback function bound to the timer handlehandle->timer_cb(handle); }}Copy the code

These callbacks are stored in a min heap. In this way, the engine only needs to judge the header element every time, and if it meets the criteria, it will take it out and execute it, until it meets a non-criteria or the queue is empty, and then it will terminate the Timer Phase.

The method in the Timer Phase to determine whether a callback meets the criteria is also simple. The message loop saves the system time each time it enters the Timer Phase, and then simply checks to see if the startup time set by the callback function in the minimum heap is longer than the saved time when it enters the Timer Phase. If so, it is pulled out for execution.

In addition, Nodejs performs a maximum number of callbacks for each Phase in each iteration of the message loop, in case one Phase has too many tasks, which will starve the subsequent phases. If the quantity exceeds, the current Phase will be forced to end and the next Phase will be entered. This rule applies to each Phase in the message loop.

4.3 Pending I/O Callback Phase

This stage is the callback function that executes your Fs. read, socket, and other IO operations, as well as various error callbacks.

4.4 Idle, Prepare Phase

It’s said to be for internal use, so we won’t discuss it too much here.

4.5 Poll Phase

This is the most important Phase in the entire message loop and waits for asynchronous requests and data. accepts new incoming connections (new socket establishment etc) and data (file read etc)). It is most important because it underpins the whole message loop mechanism.

The Poll Phase first executes the IO request in the watch_queue queue, and once the watch_queue is empty, the entire message loop goes to sleep, waiting to be awakened by a kernel event. The source code is here:

void uv__io_poll(uv_loop_t* loop, int timeout) {
  /* A series of variable initialization */
  // Determine whether an event has occurred
  if (loop->nfds == 0) {
    // Determine whether the observer queue is empty, and return if it is
    assert(QUEUE_EMPTY(&loop->watcher_queue));
    return;
  }
  
  nevents = 0;
  // The observer queue is not empty
  while(! QUEUE_EMPTY(&loop->watcher_queue)) {/* Fetch the observer object from the queue header. Fetch the event of interest to the observer object and listen. * /. W ->events = w->pevents; } assert(timeout >=- 1);
  // If there is a timeout, assign the current time to the base variable
  base = loop->time;
  // The maximum number of listener events to execute in this round
  count = 48; /* Benchmarks suggest this gives the best throughput. */
  // Enter the listening loop
  for (;; nevents = 0) {
  // If there is a timeout, initialize the spec
    if(timeout ! =- 1) {
      spec.tv_sec = timeout / 1000;
      spec.tv_nsec = (timeout % 1000) * 1000000;
    }
    
    if(pset ! =NULL)
      pthread_sigmask(SIG_BLOCK, pset, NULL);
    // Listen for kernel events and return the number of events when they arrive.
    // Timeout indicates the listening timeout period. The value will be returned once the timeout period is reached.
    // We know that timeout is the time interval for the next timers to be passed in, so the event-loop will block until the timeout or a kernel event is triggered.
    nfds = kevent(loop->backend_fd,
                  events,
                  nevents,
                  events,
                  ARRAY_SIZE(events),
                  timeout == - 1 ? NULL : &spec);

    if(pset ! =NULL)
      pthread_sigmask(SIG_UNBLOCK, pset, NULL);

    /* Update loop->time unconditionally. It's tempting to skip the update when * timeout == 0 (i.e. non-blocking poll) but there is no guarantee that the * operating system didn't reschedule our process while in the syscall. */
    SAVE_ERRNO(uv__update_time(loop));
    // If the kernel does not listen for any available events, and there is a timeout period for this listener, return.
    if (nfds == 0) { assert(timeout ! =- 1);
      return;
    }
    
    if (nfds == - 1) {
      if(errno ! = EINTR)abort(a);if (timeout == 0)
        return;

      if (timeout == - 1)
        continue;

      /* Interrupted by a signal. Update timeout and poll again. */
      gotoupdate_timeout; }...// Determine whether the observer queue for the event loop is emptyassert(loop->watchers ! =NULL);
    loop->watchers[loop->nwatchers] = (void*) events;
    loop->watchers[loop->nwatchers + 1] = (void(*)uintptr_t) nfds;
    // Loop through the event returned by the kernel, executing the event-bound callback function
    for (i = 0; i < nfds; I++) {... }}Copy the code

The uv__io_poll phase has the longest source code and the most complex logic, which can be summarized as follows: The event loop blocks in the poll phase when none of the event callbacks registered by the JS layer code return. So when you look at this, you might think, is it going to be blocked here forever? Of course Poll Phase can’t wait forever.

It has a wonderful design. In short,

  1. It first determines if the Check Phase and Close Phase have any callbacks waiting to be processed. If yes, go to the next Phase without waiting.

  2. If there are no other callbacks waiting to be executed, it sets a timeout to methods such as epoll.

Guess how much timeout is appropriate? The answer is the difference between the start time of the last callback to be executed in the Timer Phase and now, assuming the difference is detal. Because there are no callbacks waiting after the Poll Phase. So at most wait delta time, if an event wakes up the message loop, then continue to the next Phase; If nothing happens, after timeout, the message loop still moves to the next Phase, allowing the next iteration’s Timer Phase to be executed. Nodejs drives the entire message loop through Poll Phase, waiting for IO events, and arriving kernel asynchronous events.

4.6 the Check Phase

Next comes the Check Phase. This Phase deals only with the callbacks of setImmediate. So why is there a Phase here that deals with setImmediate? Simply put, because the Poll Phase Phase may set some callbacks that you want to run after the Poll Phase. So the Check Phase was added after the Poll Phase.

4.7 the Close Callbacks Phase

It deals specifically with callbacks of type CLOSE. Such as sockets. On (‘ close ‘,…). . Used to clear resources.

5 Node.js event loop section

  • Initialization of a node

    • Initialize the node environment.
    • Execute the input code.
    • Execute the process.nextTick callback.
    • Execute microtasks (Promises).

  • Enter the event loop

    • 1. The Timers phase is displayed

      • Check whether there is a timer callback in the timer queue. If yes, perform the timer callback in ascending order based on the timerId.
      • Check whether there are any process. NextTick tasks. If so, execute them all.
      • Check whether microtasks exist. If yes, execute all microtasks.
      • Exit the phase.

    • 2. Enter the IO Callbacks phase.

      • Check for pending I/O callbacks. If so, perform a callback. If not, exit the phase.
      • Check whether there are any process. NextTick tasks. If so, execute them all.
      • Check whether microtasks exist. If yes, execute all microtasks.
      • Exit the phase.

    • 3. Enter idle and prepare stage:

      These two stages have little to do with our programming.

    • 4. The poll phase is entered

      • First check to see if there are any pending callbacks, and if so, there are two cases.

      First case:

      • If there are available callbacks (available callbacks include expired timers, some IO events, etc.), perform all available callbacks.
      • Check if there is a process.nextTick callback, and if so, execute it all.
      • Check whether microtaks exist. If yes, execute all microtaks.
      • Exit the phase.

      The second case:

      • If no callback is available.

      • Check whether there is an immediate callback. If there is, exit the poll phase. If not, block at this stage and wait for new event notification.

      • Exit the poll phase if there are no callbacks that have not been completed.

    • 5. Go to the Check phase

      • If there are immediate callbacks, all immediate callbacks are executed.
      • Check if there is a process.nextTick callback, and if so, execute it all.
      • Check whether microtaks exist. If yes, execute all microtaks.
      • Exit the Check phase

    • 6. Enter the Closing phase.

      • If there are immediate callbacks, all immediate callbacks are executed.
      • Check if there is a process.nextTick callback, and if so, execute it all.
      • Check whether microtaks exist. If yes, execute all microtaks.
      • Exit closing stage

    Check if there are active handles

    • If so, continue the next cycle.
    • If not, end the event loop and exit the program.

Careful observation shows that the following processes are executed in sequence before each subphase of the event cycle exits:

  • Check if there is a process.nextTick callback, and if so, execute it all.
  • Check whether microtaks exist. If yes, execute all microtaks.
  • Exit the current phase.

6 FaQs

6.1 process. NextTick and Promise

As you can see, the callbacks for Process. NextTick and Promise are not involved in the message loop queue diagram. So what’s special about these two callbacks?

This queue guarantees all process.nextTick callbacks and then appends all Promise callbacks. Finally, at the end of each Phase, it is taken out and executed in one go.

In addition, the number of callbacks in different phases, process. NextTick, and Promise are limited. That is, if you keep adding callbacks to the queue, the entire message loop will get “stuck “. Let’s look at a picture of process. NextTick and Promise:

6.2 setTimeout (… , 0) vs. setImmediate

setTimeout(… 0)vs. setImmediate, who is fast?

Let’s do an example to get a sense of it. This is a classic FE interview question. What is the output of the following code:

// index.js

setImmediate((a)= > console.log(2))
setTimeout((a)= > console.log(1),0)
Copy the code

Answer: It could be 1, 2, or 2, 1

Let’s look at the basics of this message loop from a principle point of view. First,Nodejs starts, initializes the environment and loads our JS code (index.js). Two things happen (not yet in the message loop at this point):setImmediate adds a callback console.log(2) to the Check Phase; SetTimeout adds a callback console.log(1) to the Timer Phase. At this point, the initialization Phase is complete and the Nodejs message loop is entered, as shown below:

Why are there two outputs? The next step is crucial:

When execution reaches the Timer Phase, two things can happen. Because the system time is taken and saved when each iteration enters the Timer Phase, and the minimum unit is ms(milliseconds).

  • If the preset time of the callback in the Timer Phase > the time that the message loop is held, the callback in the Timer Phase is executed. In this case, output 1 until Check Phase executes and output 2. All in all, it’s going to be 1, 2.

  • If it is running fast, the preset time of the callback in the Timer Phase may be exactly equal to the duration of the message loop. In this case, the callback in the Timer Phase cannot be executed, and the next Phase continues. Until Check Phase, output 2. Then wait for the Timer Phase of the next iteration. At this time, the time must satisfy the preset callback time in the Timer Phase > the time saved by the message loop, so console.log(1) is executed. Output 1. All in all, it’s going to be 2, 1.

Therefore, the reason for the unstable output depends on whether the time to enter the Timer Phase is within 1ms of the time to execute the setTimeout. If you change the code to the following, you will get stable output:

require('fs').readFile('my-file-path.txt', () => {
 setImmediate((a)= > console.log(2))
 setTimeout((a)= > console.log(1))});1 / / 2
Copy the code

This is because the message loop inserts callbacks into the Timer and Check queues in the Pneding I/O Phase. In this case, according to the execution order of the message loop, the Check must be executed before the Timer

Finally, let’s look at one more interview question to further understand the Node event loop:

setImmediate((a)= > {
    console.log('setImmediate1');
    setTimeout((a)= > {
        console.log('setTimeout1')},0);
});
Promise.resolve().then(res= >{
    console.log('then');
})
setTimeout((a)= > {
    process.nextTick((a)= > {
        console.log('nextTick');
    });
    console.log('setTimeout2');
    setImmediate((a)= > {
        console.log('setImmediate2');
    });
}, 0);
//then setTimeout2 nextTick setImmediate1 setImmediate2 setTimeout1
Copy the code

Why in this order?

  • First execute the microtask Promise, outputthen;
  • Enter the timer phase, add the microtask process.nextTick callback, and outputsetTimeout2;setImmediatePut into the next stage;
  • Enter the check phase, during the phase switch outputnextTickTo complete all microtasks; Then the outputsetImmediate1.
  • Enter the next stage and output the timer’ssetImmediate1And check thesetImmediate2;

6.3 setTimeout (… , 0) can replace setImmediate

From a performance perspective, the handling of setTimeout is in the Timer Phase, where min Heap holds the Timer’s callback, so heap tuning is involved with each callback. And setImmediate just empties the queue. It will be much more efficient.

SetTimeout (… , 0) and setImmediate are entirely in two phases.

7. Event loops in the browser

The Event loop in browser is not the same as in Node. The Event loop in browser is a specification defined in HTML5, while in Node it is implemented by the Libuv library.

Event Loop in the browser All synchronous tasks are executed on the main thread, forming an execution stack in addition to the main thread, there is a task queue. The task queue is divided into macro-task(MACRO task) and micro-task(micro task).

  • Macro-task:setTimeout.setInterval.setImmediate.I/O.UI Rendering.Asynchronous event bindingSet, etc.
  • Micro-task:process.nextTick, nativePromise.MutationObserverEtc.

As can be seen from the figure, synchronous tasks will enter the JS execution stack, while asynchronous tasks will enter the callback queue for execution. Once the contents of the execution stack are completed, the waiting tasks in the task queue are read and put into the execution stack for execution.

Event loop specific process:

  • 1. In the browser, the current stack is executed first to complete the tasks in the main execution thread.

  • 2. Remove microtasks from the Microtask queue until they are empty.

  • 3. Select a task from a Macrotask and execute it.

  • 4. Check whether any Microtask exists in the Microtask. If any task exists, it is cleared.

  • Repeat 3 and 4.

So, let’s test this with an interview question. What happens when we run the following code in a browser?

setTimeout((a)= > {
    console.log('setTimeout1');
    Promise.resolve().then(data= > {
        console.log('then3');
    });
},1000);
Promise.resolve().then(data= > {
    console.log('then1');
});
Promise.resolve().then(data= > {
    console.log('then2');
    setTimeout((a)= > {
        console.log('setTimeout2');
    },1000);
});
console.log(2);
// Output: 2 then1 then2 setTimeout1 then3 setTimeout2
Copy the code

Execute the stack first, i.e., the synchronization code, so 2 is printed; Then you clear out the microtasks, so then1 then2; Since the code is executed from top to bottom, setTimeout1 is executed after 1 second; Then the microtask is cleared again, and then3 is output; Finally, the output setTimeout2 is executed

Reference:

  • Parse the nodeJS event loop
  • Nodejs message queue
  • Don’t confuse NodeJS with event loops in the browser