Why the front end should know about process communication:

Front-end domain is no longer simply written in the browser page can run, but also know electron, nodeJS, and so on, and both of these technologies require mastery of process communication.

Nodejs is a JS runtime that, unlike a browser, extends a number of apis that encapsulates the capabilities of the operating system, including process and thread apis. Learning process apis requires learning the communication mechanism between processes.

Electron is a desktop development solution based on Chromium and NodeJS. Its architecture is a main process and multiple rendering processes. These two processes also need to communicate with each other, so we need to learn the process communication mechanism of ELECTRON.

In this article we’ll take a closer look at process communication.

This article will explain the following knowledge:

  • What is a process
  • Four ways for local processes to communicate
  • What are IPC, LPC, RPC
  • How does electron do process communication
  • How do nodeJS child_process and cluster communicate
  • The nature of process communication

process

We write the code to run on the operating system, the operating system in order to better use the hardware resources, support the concurrency of multiple programs and hardware resources allocation, allocation unit is the process, the process is the program execution process. For example, record the program execution to which step, which hardware resources applied for, occupied what port.

Processes include the code to be executed, the data that the code operates on, and the Processing Control Block (PCB), because the program is the process of code execution on the data set, and the status of the execution process and the requested resources need to be recorded in a data structure (PCB). So a process consists of code, data, and PCBS.

The PCB records the PID, the address of the code executed, the status of the process (blocked, running, ready, etc.), and data structures such as semaphores, pipes, message queues, etc. used for communication.

The process is created, the code is executed, and the hardware resources (memory, hard disk files, network, etc.) are applied. It may also block in the middle, and eventually the process is destroyed. This is the lifecycle of a process.

Each process can only access its own resources. How do processes communicate with each other?

Process of communication

Different processes communicate over an intermediate medium because of the difference in available memory.

A semaphore

If it’s a simple token, represented by a number, placed in a property of the PCB, it’s called a semaphore, and an implementation of a lock, for example, can pass through a semaphore.

This semaphore idea is used a lot when we write front-end code, for example, when we implement throttling, we also need to add a marker variable.

The pipe

But semaphores can’t transmit data. There are other ways to transmit data. For example, we can communicate by reading and writing files. This is called a pipe. If the file is in memory, it is called an anonymous pipe with no name, and if it is a real disk file with a name, it is called a named pipe.

Files need to be opened, then read and written, and then closed, which is also the characteristic of pipes. Pipes are files-based encapsulation, so called because only one process can read and one process can write, and they are unidirectional (half-duplex). It also needs the target process to synchronize the consumption data, otherwise it will block.

This pipeline is very simple to implement, which is a file read and write, but can only be used for communication between two processes, only synchronous communication. The pipe method for synchronous communication is also common.

The message queue

Pipeline implementation is simple, but synchronous communication is relatively limited, so what if you want to do asynchronous communication? Just add a queue to do the buffer, that’s the message queue.

A message queue is also a communication between two processes, but it is not based on a file. Although it is one-way, it is asynchronous and can put many messages and then consume them all at once.

The Shared memory

Pipes, message queues are between two processes, what if multiple processes?

We can communicate in this way by applying for a piece of memory that multiple processes can operate on, called shared memory. All processes can read and write data to the memory, with high efficiency.

Although shared memory is efficient and can be used for communication between multiple processes, it is not all good, because multiple processes can read and write, so it is easy to mess up and control the order yourself, such as through process semaphores (marker variables).

Shared memory is suitable for communication between multiple processes and does not need to go through an intermediate medium, so it is more efficient but also more complex to use.

That’s pretty much all local processes communicate, so why add a local?

Ipc, RPC, and LPC

Inter-process Communication (IPC). Two processes may be processes of one computer or processes of different computers on the network. Therefore, Process Communication can be divided into two types:

Local procedure call (LPC) and Remote Procedure call (RPC).

Local procedure calls are the semaphores, pipes, message queues, and shared memory that we talked about above, but if they are on the network, they need to communicate through network protocols, which we actually use a lot, such as HTTP and Websocket.

So, when someone refers to IPC, they are talking about process communication, which can be classified as local or remote.

The remote ones are encapsulated based on network protocols, while the local ones are encapsulated based on semaphores, pipes, message queues, shared memory, such as electron and Nodejs, which we’ll discuss next.

Electron process communication

Electron starts the main process, then creates a rendering process through BrowserWindow and loads the HTML page for rendering. The communication between the two processes is through the IPC API provided by electron.

IpcMain, ipcRenderer

The main process listens for events through the on method of ipcMain

import { ipcMain } from 'electron';

ipcMain.on('Asynchronous Events'.(event, arg) = > {
  event.sender.send('Asynchronous event return'.'yyy');
})
Copy the code

The render process listens for events via the ON method of ipcRenderer and sends messages via send

import { ipcRenderer } from 'electron';

ipcRender.on('Asynchronous event return'.function (event, arg) {
  const message = 'Asynchronous message:${arg}`
})

ipcRenderer.send('Asynchronous Events'.'xxx')
Copy the code

The API is relatively simple to use. It is an API in the form of events that are encapsulated by the c++ layer and then exposed to js.

We can think about what kind of mechanism is it based on?

It’s obviously asynchronous, and it’s parent-child communication, so it’s done in a message queue way.

remote

In addition to the API in the form of events, electron provides an API in the form of Remote Method Invoke (RMI).

This is a further encapsulation of the message, which is to call different methods based on the message being passed, in the form of calling methods of this process, but in fact to send a message to another process to do, in the form of ipcMain, ipcRenderer essentially.

For example, in the rendering process, remote directly calls the main process’s BrowserWindow API.

const { BrowserWindow } = require('electron').remote;

let win = new BrowserWindow({ width: 800.height: 600 });
win.loadURL('https://github.com');
Copy the code

To summarize, the communication mode of electron parent and child processes is encapsulated based on message queue. There are two forms of encapsulation, one is event mode, which is used by ipcMain and ipcRenderer apis, and the other is further encapsulated into different method calls (RMI), which is also based on message. Executes a remote method but looks like it executes a local method.

nodejs

Nodejs provides an API for creating processes. There are two modules: child_process and cluster. Obviously, one is for the creation and communication of parent and child processes, and one is for multiple processes.

child_process

Child_process provides the spawn, exec, execFile, and fork apis for creating different processes:

Spawn, exec

If you want to execute commands through the shell, use spawn or exec. Because executing commands normally requires a value to be returned, the two apis differ in the way they return the value.

Spawn returns a stream, which is fetched by the data event, and exec further divides it into buffers, which are easier to use but may exceed maxBuffer.

const { spawn } = require('child_process'); 

var app = spawn('node'.'main.js' {env: {}}); app.stderr.on('data'.function(data) {
  console.log('Error:',data);
});

app.stdout.on('data'.function(data) {
  console.log(data);
});
Copy the code

In fact, exec is based on spwan encapsulation, simple scenarios can be used, sometimes need to set maxBuffer.

const { exec } = require('child_process'); 

exec('find . -type f', { maxBuffer: 1024*1024 }(err, stdout, stderr) => { 
    if (err) { 
        console.error(`exec error: ${err}`); return; 
    }   
    console.log(stdout); 
});
Copy the code

execFile

In addition to executing commands, if you want to execute executable files, use execFile’s API:

const { execFile } = require('child_process'); 

const child = execFile('node'['--version'].(error, stdout, stderr) = > { 
    if (error) { throw error; } 
    console.log(stdout); 
});
Copy the code

fork

If you want to execute js, use fork:

const { fork } = require('child_process');	

const xxxProcess = fork('./xxx.js');	
xxxProcess.send('111111');	
xxxProcess.on('message'.sum= > {	
    res.end('22222');	
});
Copy the code

summary

A quick summary of the 4 apis for child_process:

If you want to execute a shell command, use spawn and exec. Spawn returns a stream, and exec is further encapsulated as a buffer. There is no difference except that exec sometimes needs to set maxBuffer.

If you want to execute executable files, use execFile.

If you want to execute the js file, use fork.

Child_process process communication

The child process created by child_process communicates with the parent process, which is ipc.

pipe

First, there is support for pipes, which are obviously encapsulated by the mechanism of pipes to synchronously transmit streams of data.

const { spawn } = require('child_process'); 

const find = spawn('cat'['./aaa.js']);
const wc = spawn('wc'['-l']);  find.stdout.pipe(wc.stdin);
Copy the code

For example, to pipe the output stream of one process to the input stream of another process, the following shell command has the same effect:

cat ./aaa.js | wc -l
Copy the code

message

Spawn supports the stdio parameter, which can be set to the parent process’s stdin, stdout, or stderr, such as pipe or null. There is a fourth parameter, ipc, which is delivered by event, obviously based on message queues.

const { spawn } = require('child_process');

const child = spawn('node'['./child.js'] and {stdio: ['pipe'.'pipe'.'pipe'.'ipc']}); child.on('message'.(m) = > { 
    console.log(m); 
}); 
child.send('xxxx');
Copy the code

The child process created by the fork API has ipc’s messaging mechanism, which can be used directly.

const { fork } = require('child_process');	

const xxxProcess = fork('./xxx.js');	
xxxProcess.send('111111');	
xxxProcess.on('message'.sum= > {	
    res.end('22222');	
});
Copy the code

cluster

Cluster is no longer a parent and child process, but more processes and provides an API for fork.

For example, the HTTP server starts multiple processes to process requests based on the number of cpus.

import cluster from 'cluster';
import http from 'http';
import { cpus } from 'os';
import process from 'process';

const numCPUs = cpus().length;

if (cluster.isPrimary) {
  for (let i = 0; i < numCPUs; i++) { cluster.fork(); }}else {
  const server = http.createServer((req, res) = > {
    res.writeHead(200);
    res.end('hello world\n');
  })
  
  server.listen(8000);
  
  process.on('message'.(msg) = > {
    if (msg === 'shutdown') { server.close(); }}); }Copy the code

It also supports an API in the form of events for messaging between multiple processes, which are based on message queues because multiple processes communicate only with parent processes, and children cannot communicate directly with each other.

The Shared memory

The communication between the child process also has to pass through the parent process once, to read and write the message queue many times, is too inefficient, can not directly share memory?

Currently nodeJS is not supported. It can be implemented using the third party package shm-typed array.

www.npmjs.com/package/shm…

conclusion

Process includes code, data and PCB, which is the process of one execution of the program. PCB records various information in the execution process, such as allocated resources, executed address, data structure used for communication, etc.

Processes need to communicate with each other through semaphores, pipes, message queues, shared memory.

  • A semaphore is a simple numeric marker that does not transmit specific data.

  • Pipes are based on the idea of a file, where one process writes and the other reads, and are synchronous, suitable for both processes.

  • The message queue has a buffer and can process messages asynchronously, suitable for both processes.

  • Shared memory allows multiple processes to directly operate on the same memory segment. It is applicable to multiple processes, but the order of access is controlled.

These four methods are used by local processes to communicate. Network processes can also communicate based on network protocols.

Process communication is called IPC, local is called LPC, and remote is called RPC.

Encapsulating the message in a layer of concrete method calls, called RMI, would be as effective as executing another process’s method in this process.

Electron and NodeJS are both packages based on the above operating system mechanism:

  • Elctron supports the ipcMain and ipcRenderer messaging modes, and supports the remote RMI mode.

  • Nodejs has two modules, child_process and cluster, which are related to processes. Child_process is between a child and a child process, and cluster is multiple processes:

    • Child_process provides spawn, exec to execute shell commands, execFile to execute executable files, and fork to execute JS. Two IPC modes are provided: PIPE and Message.

    • The cluster also provides a fork that provides communication in the form of message.

Of course, no matter what form of encapsulation is, the operating system provides semaphores, pipes, message queues, shared memory these four mechanisms.

Ipc is a frequently encountered requirement in development, and hopefully this article will help you sort through the hierarchy of encapsulation from the operating system layer to different languages and runtimes.