Log4js log loss in PM2 Cluster mode

Node application is a single thread application. Although JS can use asynchronous I/O to avoid thread blocking, it cannot use the advantage of multi-core CPU to improve operation efficiency, and multi-thread is still needed to improve throughput. Node Cluster can generate multiple worker threads sharing the same TCP connection. The main thread communicates with the worker thread through the IPC channel and uses round-robin load balancing to handle inter-thread pressure.

PM2 Cluster makes it easier for nodes to operate clusters. PM2 generates worker threads based on the number of server CPU cores. You only need to start the application in the following way:

pm2 start app.js -i 0
Copy the code

But when the PM2 Cluster collided with Log4js, it created a crater into which I stepped.

One day, the server reported abnormal log of request parameters on the frontline. In order to trace the cause of the anomaly, I searched all online servers and failed to find relevant logs. Server-side exception logs are not fabricated, and the loss of front-end logs is not accidental. In order to count the loss rate of logs, we launched 100 requests in offline environment quantitatively, only 25 logs were generated. Many experiments found that the loss rate was stable at 3/4, which was appalling! Hot (good) love (strange) technology (heart) art (heavy) I looked up Log4js source code:

configuration.addListener((config) = > {
    // clear out the listeners, because configure has been called.
    listeners.length = 0;

    disabled = config.disableClustering;
    pm2 = config.pm2;
    pm2InstanceVar = config.pm2InstanceVar || 'NODE_APP_INSTANCE';

    debug(`clustering disabled ? ${disabled}`);
    debug(`cluster.isMaster ? ${cluster.isMaster}`);
    debug(`pm2 enabled ? ${pm2}`);
    debug(`pm2InstanceVar = ${pm2InstanceVar}`);
    debug(`process.env[${pm2InstanceVar}] = ${process.env[pm2InstanceVar]}`);

    // just in case configure is called after shutdown.
    if (pm2) {
        process.removeListener('message', receiver);
    }
    if (cluster.removeListener) {
        cluster.removeListener('message', receiver);
    }

    if (config.disableClustering) {
        debug('Not listening for cluster messages, because clustering disabled.');
    } else if (isPM2Master()) {
        // PM2 cluster support
        // PM2 runs everything as workers - install pm2-intercom for this to work.
        // we only want one of the app instances to write logs.
        debug('listening for PM2 broadcast messages');
        process.on('message', receiver);
    } else if (cluster.isMaster) {
        debug('listening for cluster messages');
        cluster.on('message', receiver);
    } else {
        debug('not listening for messages, because we are not a master process.'); }});Copy the code

Please note:

PM2 runs everything as workers – install pm2-intercom for this to work.

Log4js In Cluster mode, the worker sends the log to the master, and the master writes the log to the file. However, in PM2 Cluster mode, all processes are worker:

Install the PM2-Intercom interprocess communication module as instructed by Log4js:

IsPM2Master () :

const isPM2Master = (a)= > pm2 && process.env[pm2InstanceVar] === '0';
const isMaster = (a)= > disabled || cluster.isMaster || isPM2Master();
Copy the code

Log4js configure pm2 and pm2InstanceVar for isPM2Master.

Log4JS.configure({
    // ...
    pm2: true.pm2InstanceVar: 'INSTANCE_ID'
});
Copy the code

Log4js log loss in PM2 Cluster mode is resolved.

---

Add:

Implement Node Cluster:

const OS = require('os');
const Cluster = require('cluster');
const Koa = require('koa');
const App = new Koa();
if (Cluster.isMaster) {
    for (let i = 0; i < OS.cpus().length; i++) Cluster.fork();
    console.log('master', process.pid);
} else {
    App.listen(3000);
    console.log('worker', process.pid);
}
Copy the code

The relationship between port PID and PID List displayed on the console:

Start the Node application using the PM2 Cluster. The relationship between port PID and PID List displayed on the PM2 console is as follows:

---

Author: Silly love kitten

My garden: sunmengyuan. Making. IO/garden /

My Github: github.com/sunmengyuan

The original link: sunmengyuan. Making. IO/garden / 2018…