Deep into Nodejs source code to explore the CPU information acquisition and utilization calculation

On Linux you can see the current CPU utilization using the top or htop command, and you may have seen it in some monitoring tools. How is it calculated? How do we do that in Nodejs?

With these questions in mind, this section will start with a brief introduction to CPU utilization under Linux, and then delve into Nodejs source code to discuss how to obtain CPU information and calculate CPU utilization at a certain time.

To begin, take a look at a diagram that shows the entire process call of the Nodejs OS module to read system CPU information, which you’ll see again in more detail below.

CPU usage in Linux

The CPU usage in Linux is divided into user state (execution time in user mode), system state (kernel execution time), and idle state (idle system process execution time). The sum of the three is the total CPU execution time. You can view the CPU activity information in the /proc/stat file.

CPU utilization is the total execution time of non-idle processes/CPUS.

> cat /proc/statcpu 2255 34 2290 22625563 6290 127 456 cpu0 1132 34 1441 11311718 3675 127 438 cpu1 1123 0 849 11313845 2614 0 18 intr 114930548 113199788 3 0 5 263 0 4 [... lots more numbers ...]  ctxt 1990473# Number of context swaps that have occurred in the CPU since system startup
btime 1062191376 # Time to date, in seconds
processes 2915 # Number of tasks created since system startup
procs_running 1 # Number of tasks in the current run queue
procs_blocked 0 # Number of tasks currently blocked
Copy the code

The first line indicates the total CPU usage. Cpu0 and CPU1 indicate the number of CPU cores (CPU0 + CPU1 + cpuN = the total number of CPU cores).

• User: indicates the CPU time (unit: jiffies) accumulated in user mode since system startup, excluding the process whose NICE value is negative.
• Nice: indicates the CPU time occupied by the process whose NICE value is negative since the system starts.
• System: indicates the core time accumulated from the system startup to the current time
• Idle: indicates the wait time accumulated from the system startup to the current time, excluding the DISK I/O wait time
• Iowait: indicates the I/O wait time accumulated from the system startup to the current time
• Irq: hard interrupt time accumulated from system startup to the current time
• Softirq: indicates the soft interrupt time accumulated from the system startup to the current time

About the/proc/stat, reference here www.linuxhowtos.org/System/proc…

Formula of CPU usage in a specified period

The /proc/stat file shows the total CPU time accumulated from startup to current. To calculate the CPU usage in a certain period, you need to calculate the CPU usage at t1 and T2.

CPU execution time from T1 to T2:

t1 = (user1 + nice1 + system1 + idle1 + iowait1 + irq1 + softirq1)
t2 = (user2 + nice2 + system2 + idle2 + iowait2 + irq2 + softirq2) 
t = t2 - t1
Copy the code

Idle CPU usage time range T1 to T2:

idle = (idle2 - idle1)
Copy the code

CPU idle rate between T1 and T2:

idleRate = idle / t;
Copy the code

CPU usage from T1 to T2:

usageRate = 1 - idleRate;
Copy the code

The formula for calculating the CPU usage in a certain period of time can be understood first, and Nodejs will be used in the end.

This section can be extended, and those interested can try using shell scripts to calculate CPU utilization.

How do I get CPU information in Nodejs?

The Nodejs OS module cpus() method returns an array of objects containing information about each logical CPU kernel.

Question: how exactly did you get this data? Is this related to /proc/stat under Linuv? With these questions can only be seen from the source code.

const os = require('os');

os.cpus();
Copy the code

1. JS layers

The lib module is the js layer module code exposed by Node.js. In the os.js file, only the cpus related core code is reserved, where getCPUs is imported via internalBinding(‘ OS ‘).

InternalBinding is the bridge between the JS layer and the C++ layer.

// https://github.com/Q-Angelo/node/blob/master/lib/os.js#L41
const {
  getCPUs,
  getFreeMem,
  getLoadAvg,
  ...
} = internalBinding('os');

// https://github.com/Q-Angelo/node/blob/master/lib/os.js#L92
function cpus() {
  // [] is a bugfix for a regression introduced in 51cea61
  const data = getCPUs() || [];
  const result = [];
  let i = 0;
  while (i < data.length) {
    result.push({
      model: data[i++],
      speed: data[i++],
      times: {
        user: data[i++],
        nice: data[i++],
        sys: data[i++],
        idle: data[i++],
        irq: data[i++]
      }
    });
  }
  return result;
}

// https://github.com/Q-Angelo/node/blob/master/lib/os.js#L266
module.exports = {
  cpus,
  ...
};
Copy the code

2. C + + layer

2.1 the Initialize:

SRC /node_os.cc file has an Initialize operation. The SRC /node_os.cc file has an Initialize operation. GetCPUs corresponds to the GetCPUInfo method, which is implemented next.

// https://github.com/Q-Angelo/node/blob/master/src/node_os.cc#L390
void Initialize(Local<Object> target,
                Local<Value> unused,
                Local<Context> context,
                void* priv) {
  Environment* env = Environment::GetCurrent(context);
  env->SetMethod(target, "getCPUs", GetCPUInfo); . target->Set(env->context(), FIXED_ONE_BYTE_STRING(env->isolate(),"isBigEndian"),
              Boolean::New(env->isolate(), IsBigEndian())).Check();
}
Copy the code

2.2 GetCPUInfo Implementation:

• The core of the uv_cpu_info method is to pass the pointer to &cpu_infos and &count to get the CPU information and the number count
• The for loop iterates through each CPU core, assigning the variable CI to user, nice, sys… This data is familiar, and is retrieved in Nodejs using os.cpus(), and is stored in the Result object
• When the traversal is complete, use uv_free_CPU_info to reclaim cpu_INFOS and count
• Finally, set the parameter Array::New(isolate, result.data(), result.size()) to be returned as an Array.

// https://github.com/Q-Angelo/node/blob/master/src/node_os.cc#L113
static void GetCPUInfo(const FunctionCallbackInfo<Value>& args) {
  Environment* env = Environment::GetCurrent(args);
  Isolate* isolate = env->isolate();

  uv_cpu_info_t* cpu_infos;
  int count;

  int err = uv_cpu_info(&cpu_infos, &count);
  if (err)
    return;

  // It's faster to create an array packed with all the data and
  // assemble them into objects in JS than to call Object::Set() repeatedly
  // The array is in the format
  // [model, speed, (5 entries of cpu_times), model2, speed2, ...]
  std: :vector<Local<Value>> result(count * 7);
  for (int i = 0, j = 0; i < count; i++) {
    uv_cpu_info_t* ci = cpu_infos + i;
    result[j++] = OneByteString(isolate, ci->model);
    result[j++] = Number::New(isolate, ci->speed);
    result[j++] = Number::New(isolate, ci->cpu_times.user);
    result[j++] = Number::New(isolate, ci->cpu_times.nice);
    result[j++] = Number::New(isolate, ci->cpu_times.sys);
    result[j++] = Number::New(isolate, ci->cpu_times.idle);
    result[j++] = Number::New(isolate, ci->cpu_times.irq);
  }

  uv_free_cpu_info(cpu_infos, count);
  args.GetReturnValue().Set(Array::New(isolate, result.data(), result.size()));
}
Copy the code

3. Libuv layer

One of the most important methods, uv_cpu_info, is used to retrieve the data source, and now it’s time to find it

3.1 node_os. Cc:

The built-in module node_os.cc references the header env-inl.h

// https://github.com/Q-Angelo/node/blob/master/src/node_os.cc#L22
#include "env-inl.h".Copy the code

3.2 env – inl. H:

Uv.h is also referenced in env-inl. H

// https://github.com/Q-Angelo/node/blob/master/src/env-inl.h#L31
#include "uv.h"
Copy the code

3.3 uv. H:

H (header file) contains the declaration of the member and method in the class. It does not contain the concrete implementation. The declaration is found.

In addition to uv_cpu_info, the uv_free_cpu_info method is also declared. The corresponding method is mainly used for recycling. The cpu_infos and count parameters are reclaimed by the C++ layer at the end of the data traversal.

/* https://github.com/Q-Angelo/node/blob/master/deps/uv/include/uv.h#L1190 */
UV_EXTERN int uv_cpu_info(uv_cpu_info_t** cpu_infos, int* count);
UV_EXTERN void uv_free_cpu_info(uv_cpu_info_t* cpu_infos, int count);
Copy the code

The Libuv layer is just an encapsulation of the underlying operating system. The following is the implementation of the operating system layer.

4. OS layer

The 4.1 Linux – core. C:

A search for uv_cpu_info under DEps /uv/ will reveal that it has many implementations of AIX, Cygwin.c, Darwin.c, freebsd. C, Linux-core-c, and more. Linux-core. c seems to be the implementation of Linux, so let’s focus on the implementation.

The uv__open_file(“/proc/stat”) parameter /proc/stat is the location of the CPU information under Linux.

// https://github.com/Q-Angelo/node/blob/master/deps/uv/src/unix/linux-core.c#L610

int uv_cpu_info(uv_cpu_info_t** cpu_infos, int* count) {
  unsigned int numcpus;
  uv_cpu_info_t* ci;
  int err;
  FILE* statfile_fp;

  *cpu_infos = NULL;
  *count = 0;

  statfile_fp = uv__open_file("/proc/stat"); . }Copy the code

4.2 the core. C:

The ultimate implementation of the uv__open_file() method is found in the /deps/uv/ SRC/Unix /core.c file, which gets a pointer to a file in read-only and execute closed mode.

On Linux, the Nodejs OS module’s cpus() method also ends up reading the /proc/stat file to get CPU information.

// https://github.com/Q-Angelo/node/blob/master/deps/uv/src/unix/core.c#L455
/* get a file pointer to a file in read-only and close-on-exec mode */
FILE* uv__open_file(const char* path) {
  int fd;
  FILE* fp;

  fd = uv__open_cloexec(path, O_RDONLY);
  if (fd < 0)
    return NULL;

   fp = fdopen(fd, "r");
   if (fp == NULL)
     uv__close(fd);

   return fp;
}
Copy the code

When should I locate the win directory? When to locate the Unix directory?

This depends on the Libuv layer. In the book “Nodejs in Depth”, there is a paragraph that “Node will determine the platform conditions during compilation and selectively compile source files in Unix or WIN directories into the target program”, so this is determined at compile time rather than run time.

A picture is worth a thousand words

By combing the CPU information reading process of OS module, the classic Nodejs architecture is shown again:

JavaScript -> internalBinding -> C++ -> Libuv -> OS

Practice in Nodejs

After understanding the above principles, it is easy to implement in Nodejs. The system layer provides us with perfect API calls.

Os.cpus () data indicator

Nodejs os.cpus() returns an array of objects with a times field that contains user, nice, sys, idle, and IRq, representing the number of milliseconds spent in user mode, good mode, system mode, idle mode, and interrupt mode respectively. Compared to Linux, it is more intuitive to view it directly through cat /proc/stat.

[{model: 'Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz'.speed: 2926.times: {
      user: 252020.nice: 0.sys: 30340.idle: 1070356870.irq: 0}}...Copy the code

Coding practices in Nodejs

Define method _getCPUInfo to obtain system CPU information.

The getCPUUsage method provides “real time” monitoring of CPU utilization. This “real time” is not absolute real time, and there is always a time lag. The default value of 1 second in our implementation below can be adjusted using options.ms.

const os = require('os');
const sleep = ms= > new Promise(resolve= > setTimeout(resolve, ms));

class OSUtils {
  constructor() {
    this.cpuUsageMSDefault = 1000; // Default CPU usage period
  }

  @param {Number} options. ms [time range, default: 1000ms, Which is 1 seconds] * @ param {Boolean} Options. The percentage with [true (in percentage results back) | false] * @ returns {Promise} * /
  async getCPUUsage(options={}) {
    const that = this;
    let { cpuUsageMS, percentage } = options;
    cpuUsageMS = cpuUsageMS || that.cpuUsageMSDefault;
    const t1 = that._getCPUInfo(); // Indicates the CPU information at the t1 time point

    await sleep(cpuUsageMS);

    const t2 = that._getCPUInfo(); // CPU information at t2 time
    const idle = t2.idle - t1.idle;
    const total = t2.total - t1.total;
    let usage = 1 - idle / total;

    if (percentage) usage = (usage * 100.0).toFixed(2) + "%";

    return usage;
  }

  Returns {Object} CPU information */
  _getCPUInfo() {
    const cpus = os.cpus();
    let user = 0, nice = 0, sys = 0, idle = 0, irq = 0, total = 0;

    for (let cpu in cpus) {
      const times = cpus[cpu].times;
      user += times.user;
      nice += times.nice;
      sys += times.sys;
      idle += times.idle;
      irq += times.irq;
    }

    total += user + nice + sys + idle + irq;

    return {
      user,
      sys,
      idle,
      total,
    }
  }
}
Copy the code

The usage mode is as follows:

const cpuUsage = await osUtils.getCPUUsage({ percentage: true });
console.log('CPU Usage: ', cpuUsage) // CPU usage: 13.72%
Copy the code

conclusion

This article first from Linux CPU utilization concept to do a simple explanation, then in-depth Nodejs OS module source code to obtain system CPU information comb. On the other hand, it also presents the classic Nodejs framework JavaScript -> internalBinding -> C++ -> Libuv -> OS. This is common to other apis and can be used as a reference. Finally, Nodejs is used to calculate the CPU utilization.

Reference

• www.penglixun.com/tech/system…
• Blog.csdn.net/htjx99/arti…
• www.linuxhowtos.org/System/proc…
• Github.com/Q-Angelo/no…
• nodejs.cn/api/os.html