>> Blog post
Some of the tool methods implemented in this paper are in the early/testing stage and are still being continuously optimized for reference only…
Developed/tested on Ubuntu20.04, available directly for the Electron project, test version: [email protected]/9.3.5
Contents
├─ Contents (You Are Here!) │ ├── II. Structure Diagram │ ├─ III. What can be done with the electron- Re? │ │ ├ ─ ─ 1) used for Electron application └ ─ ─ 2) application for Electron/Nodejs │ ├ ─ ─ IV. The UI features introduced │ ├ ─ ─ main interface │ ├ ─ ─ function 1: Kill process │ ├ ─ ─ function 2: └ │ ├─ ├─ │ ├─ ├─ │ ├─ │ ├─ │ ├─ │ ├─ │ ├─ │ ├─ │ ├─ │ ├─ │ ├─ │ ├─ Use & Principles │ ├─ Introduce │ ├─ How to catch Process Resource occupation? │ ├─ How to share data between main process and UI? ├ ─ How to create line Charts in THE UI window? │ ├─ VI. Known Problems │ ├─ VII. Next To Do │ ├─ VIII. Several practical application examples │ ├ ─ ─ 1) Service/MessageChannel example │ ├ ─ ─ 2) ChildProcessPool/ProcessHost example │ └ ─ ─ 3)testTest directory ExampleCopy the codeI. introduction
Recently, I encountered some performance problems when making a parallel multi-file fragment upload module (based on Electron and React), which are mainly reflected in: When a large number of files (1000-10000) are added to the front end at the same time for parallel uploading (the number of files uploaded at the same time is 6 by default), the whole application window is stuck without lazy loading optimization. Therefore, we studied the Electron/Nodejs multi-process and tried to optimize the upload process with the multi-process architecture.
At the same time, a tool for conducting Electron/Node multi-process management and call has also been written, which is released as NPM component and can be installed directly:
> > making address
$: npm install electron-re --save
# or
$: yarn add electron-re
Copy the codeThe previous article “Electron/Node Multi-process Tool Development Diary” describes the development background, problem scenarios, and detailed application methods of Electron – Re. This article does not explain the basic application of Electron – Re, but mainly introduces the development and application of the new feature multi-process management UI. UI interface based on electron – re existing BrowserService/MessageChannel and ChildProcessPool/ProcessHost infrastructure drive, using React17 / Babel7 development, main interface:
II. Electron – Re architecture diagram
III. What can be done with electron- Re?
1. For Electron application
BrowserServiceMessageChannel
Some of the best practices in Electron suggest putting CPU-hogging code in the render process rather than directly in the main process. Here’s a look at the Chromium architecture:
Each renderer process has a global object, RenderProcess, which manages communication with the parent browser process and maintains a global state. The browser process maintains a RenderProcessHost object for each renderer process to manage browser state and communicate with the renderer process. Browser processes and renderers communicate using Chromium’s IPC system. In Chromium, UIprocess needs IPC synchronization with main process continuously during page rendering. If main process is busy, UIprocess will block during IPC. Therefore, if the main process continues to consume CPU time or block the task of synchronous IO, it will be blocked to a certain extent, thus affecting the IPC communication between the main process and each render process. If the IPC communication is delayed or blocked, the render process window will be stuck and drop frames, or even frozen in serious cases.
Therefore, based on the existing logic of electron’s Main Process and Renderer Process, an independent Service concept is developed. A Service is a background process that does not need to display an interface. It does not participate in UI interaction and provides services for the main process or other rendering processes. Its underlying implementation is a rendering window process that allows Node injection and remote calls.
This allows you to write cpu-consuming operations in your code (such as maintaining a queue of thousands of upload tasks in a file upload) into a single JS file, and then use the BrowserService constructor to construct a Service instance that takes the address path of this JS file, thus separating them from the main process. If you say that this part of the CPU operation directly into the render window process can? This depends on the architectural design of the project itself and the trade-offs between data transfer performance loss and transfer time between processes. Create a simple example of a Service:
const { BrowserService } = require('electron-re');
const myServcie = new BrowserService('app', path.join(__dirname, 'path/to/app.service.js'));
Copy the codeIf BrowserService is used, to send messages between the main process, the renderer process, and the service process arbitrarily, you need to use the MessageChannel provided by electron -Re. Its interface design is basically the same as the IPC built into electron. Also based on the IPC communication principle to achieve, a simple example is as follows:
/* ---- main.js ---- */
const { BrowserService } = require('electron-re');
// The main process sends a message to a service-app
MessageChannel.send('app'.'channel1', { value: 'test1' });
Copy the code2. Apply it to the Electron/Nodejs application
ChildProcessPoolProcessHost
Additionally, if you want to create child processes that are not dependent on the Electron runtime (see Nodejs child_process), you can use the ChildProcessPool class provided with Electron – Re, a process pool written specifically for the NodeJS runtime. Because the process itself is expensive to create, use a process pool to reuse already created child processes to maximize the performance benefits of a multi-process architecture, as shown in the following simple example:
const { ChildProcessPool } = require('electron-re');
global.ipcUploadProcess = new ChildProcessPool({
path: path.join(app.getAppPath(), 'app/services/child/upload.js'), max: 6
});
Copy the codeIn general, in our child process execution file (the script specified by the path parameter when creating the child process), if we want to synchronize data between the main and child processes, This can be done using process.send(‘channel’, params) and process.on(‘channel’, function) (if the process is forked or ipc communication is manually enabled). But this also forces us to focus on the communication between processes while dealing with business logic. You need to know when the child process is finished, and then use process.send to send data back to the main process, which is cumbersome.
Electron – Re introduces the concept of ProcessHost, which I call “process transaction center.” In practice, you only need to register each task function as multiple monitored transactions through processhost.registry (‘task-name’, function) in the child process execution file. Childprocesspool. send(‘task-name’, params) triggers the child process’s transaction logic. Childprocesspool. send() returns a Promise instance to fetch the callback data. A simple example is as follows:
/* -- in main process -- */.global.ipcUploadProcess
.send('task1', params)
.then(rsp= > console.log(rsp));
/* -- child process -- */
const { ProcessHost } = require('electron-re');
ProcessHost
.registry('task1'.(params) = > {
return { value: 'task-value' };
})
.registry('init-works'.(params) = > {
return fetch(url);
});
Copy the codeIV. UI function introduction
II describes the main functions of electron- Re, based on which the MULTI-process monitoring UI panel is implemented
The main interface
UI refer to the electron- process-Manager design
Preview:
The main functions are as follows:
-
Displays all the processes started in Electron, including the main process, the normal render process, the Service process (introduced by Electron -re), and the child processes created by ChildProcessPool (introduced by Electron – Re).
-
The process list displays the process ID, process ID, parent process ID, memory usage, and CPU usage of each process. The process ids are as follows: Main, Service, renderer, node. Click on the table header to increment/decrement an item.
-
After selecting a process, you can Kill the process, view the Console data of the process Console, and view the CPU/ memory usage trend of the process within one minute. If the process is a rendering process, you can also use the DevTools button to open the built-in debugging tool.
-
Child processes created by ChildProcessPool do not support direct DevTools debugging, but because of the –inspect parameter, you can use Chrome ://inspect for remote debugging.
Function 1: Kill process
Function 2: Enable DevTools in one click
Function 3: View process logs
Function 3: View the CPU/Memory usage trend of processes
V. Use & principle
The introduction of
- Introduce in the Electron main process entry file:
const {
MessageChannel, // must required in main.js even if you don't use it
ProcessManager
} = require('electron-re');
Copy the code- The process management window IS displayed
ProcessManager.openWindow();
Copy the codeHow do I capture process resource usage?
1. Use ProcessManager to listen for multiple process numbers
- 1) Place the window process ID into the ProcessManager listening list in the Electron window create event
/* --- src/index.js --- */. app.on('web-contents-created'.(event, webContents) = > {
webContents.once('did-finish-load'.() = > {
const pid = webContents.getOSProcessId();
if (
exports.ProcessManager.processWindow &&
exports.ProcessManager.processWindow.webContents.getOSProcessId() === pid
) { return; }
exports.ProcessManager.listen(pid, 'renderer');
webContents.once('closed'.function(e) {
exports.ProcessManager.unlisten(this.pid); }.bind({ pid })); . })});Copy the code- 2) Add the process ID to the listener list when the process pool forks a child process
/* --- src/libs/ChildProcessPool.class.js --- */.const { fork } = require('child_process');
class ChildProcessPool {
constructor({ path, max=6, cwd, env }){...this.event = new EventEmitter();
this.event.on('fork'.(pids) = > {
ProcessManager.listen(pids, 'node');
});
this.event.on('unfork'.(pids) = > {
ProcessManager.unlisten(pids);
});
}
/* Get a process instance from the pool */
getForkedFromPool(id="default") {
letforked; . forked = fork(this.forkedPath, ...) ;this.event.emit('fork'.this.forked.map(fork= >fork.pid)); .returnforked; }... }Copy the code- 3) Listen for process IDS during Service process registration
When BrowserService is created, it sends a Registry request to MessageChannel to register a Service globally. In this case, put the process ID in the listener list:
/* --- src/index.js --- */.exports.MessageChannel.event.on('registry'.({pid}) = > {
exports.ProcessManager.listen(pid, 'service'); }); .exports.MessageChannel.event.on('unregistry'.({pid}) = > {
exports.ProcessManager.unlisten(pid)
});
Copy the code2. Collect process load data once per second using pidUsage library compatible with multiple platforms:
/* --- src/libs/ProcessManager.class.js --- */.const pidusage = require('pidusage');
class ProcessManager {
constructor() {
this.pidList = [process.pid];
this.typeMap = {
[process.pid]: 'main'}; . }/* -------------- internal -------------- */
/* Set up external library collection and send to UI process */
refreshList = () = > {
return new Promise((resolve, reject) = > {
if (this.pidList.length) {
pidusage(this.pidList, (err, records) = > {
if (err) {
console.log(`ProcessManager: refreshList -> ${err}`);
} else {
this.processWindow.webContents.send('process:update-list', { records, types: this.typeMap });
}
resolve();
});
} else{ resolve([]); }}); }/* Set timer for collection */
setTimer() {
if (this.status === 'started') return console.warn('ProcessManager: the timer is already started! ');
const interval = async() = > {setTimeout(async() = > {await this.refreshList()
interval(this.time)
}, this.time)
}
this.status = 'started';
interval()
}
...
Copy the code3. Listen to process output to collect process logs
Child processes created by the process pool can collect logs by listening to the STdOUT standard output stream. The Electron rendering window process can be collected by listening on the IPC communication event console-message.
/* --- src/libs/ProcessManager.class.js --- */
class ProcessManager {
constructor(){... }/* pipe to process.stdout */
pipe(pinstance) {
if (pinstance.stdout) {
pinstance.stdout.on(
'data'.(trunk) = > {
this.stdout(pinstance.pid, trunk); }); }}... }/* --- src/index.js --- */
app.on('web-contents-created'.(event, webContents) = > {
webContents.once('did-finish-load'.() = > {
constpid = webContents.getOSProcessId(); . webContents.on('console-message'.(e, level, msg, line, sourceid) = > {
exports.ProcessManager.stdout(pid, msg); }); . })});Copy the codeHow do you share data between the main process and the UI?
Based on Electron native IPC asynchronous communication
1. Use ProcessManager to send log data to the UI rendering window
Console data for all processes collected within 1 second is temporarily cached in an array, sent to the UI process once a second by default, and then emptied.
Note that the child processes in ChildProcessPool are created using the child_process.fork() method of Node.js, which generates the shell, and the stdio argument is specified as ‘pipe’. Specifies that a pipe should be created between the child and its parent so that the parent can listen directly for the stdout.on(‘data’) event on the child object to get the child’s standard output stream.
/* --- src/libs/ProcessManager.class.js --- */
class ProcessManager {
constructor(){... }/* pipe to process.stdout */
pipe(pinstance) {
if (pinstance.stdout) {
pinstance.stdout.on(
'data'.(trunk) = > {
this.stdout(pinstance.pid, trunk); }); }}/* send stdout to ui-processor */
stdout(pid, data) {
if (this.processWindow) {
if (!this.callSymbol) {
this.callSymbol = true;
setTimeout(() = > {
this.processWindow.webContents.send('process:stdout'.this.logs);
this.logs = [];
this.callSymbol = false;
}, this.time);
} else {
this.logs.push({ pid: pid, data: String.prototype.trim.call(data) }); }}}... }Copy the code2. Use ProcessManager to send process load information to the UI rendering window
/* --- src/libs/ProcessManager.class.js --- */
class ProcessManager {
constructor(){... }/* Set up external library collection and send to UI process */
refreshList = () = > {
return new Promise((resolve, reject) = > {
if (this.pidList.length) {
pidusage(this.pidList, (err, records) = > {
if (err) {
console.log(`ProcessManager: refreshList -> ${err}`);
} else {
this.processWindow.webContents.send('process:update-list', { records, types: this.typeMap });
}
resolve();
});
} else{ resolve([]); }}); }... }Copy the code3. The UI window will process and temporarily store the data after getting it
import { ipcRenderer, remote } from 'electron'; . ipcRenderer.on('process:update-list'.(event, { records, types }) = > {
console.log('update:list');
const { history } = this.state;
for (let pid in records) {
history[pid] = history[pid] || { memory: [].cpu: []};if(! records[pid])continue;
history[pid].memory.push(records[pid].memory);
history[pid].cpu.push(records[pid].cpu);
// Store the latest 60 process load data
history[pid].memory = history[pid].memory.slice(-60);
history[pid].cpu = history[pid].cpu.slice(-60);
}
this.setState({
processes: records,
history,
types
});
});
ipcRenderer.on('process:stdout'.(event, dataArray) = > {
console.log('process:stdout');
const { logs } = this.state;
dataArray.forEach(({ pid, data }) = > {
logs[pid] = logs[pid] || [];
logs[pid].unshift(` [The ${new Date().toLocaleTimeString()}] :${data}`);
});
// Store the last 1000 log outputs
Object.keys(logs).forEach(pid= > {
logs[pid].slice(0.1000);
});
this.setState({ logs });
});
Copy the codeHow to draw a line chart in a UI window?
1. Note that with react. PureComponent, shallow comparisons are automatically made in property updates to reduce unnecessary rendering
/* *************** ProcessTrends *************** */
export class ProcessTrends extends React.PureComponent {
componentDidMount(){... }...render() {
const { visible, memory, cpu } = this.props;
if (visible) {
this.uiDrawer.draw();
this.dataDrawer.draw(cpu, memory);
};
return (
<div className={`process-trends-containerThe ${!visible ? 'hidden' : 'progressive-show'} `} >
<header>
<span className="text-button small" onClick={this.handleCloseTrends}>X</span>
</header>
<div className="trends-drawer">
<canvas
width={document.body.clientWidth * window.devicePixelRatio}
height={document.body.clientHeight * window.devicePixelRatio}
id="trendsUI"
/>
<canvas
width={document.body.clientWidth * window.devicePixelRatio}
height={document.body.clientHeight * window.devicePixelRatio}
id="trendsData"
/>
</div>
</div>)}}Copy the code2. Use two canvases to draw coordinate axes and line segments respectively
To overlap the two canvases as much as possible to ensure that the static axes will not be drawn repeatedly, we need to initialize an axis drawing object uiDrawer and a data line drawing object dataDrawer after the component is mounted
.componentDidMount() {
this.uiDrawer = new UI_Drawer('#trendsUI', {
xPoints: 60.yPoints: 100
});
this.dataDrawer = new Data_Drawer('#trendsData');
window.addEventListener('resize'.this.resizeDebouncer); }...Copy the codeHere are the basic Canvas related drawing commands:
this.canvas = document.querySelector(selector);
this.ctx = this.canvas.getContext('2d');
this.ctx.strokeStyle = lineColor; // Set the line segment color
this.ctx.beginPath(); // Create a new path
this.ctx.moveTo(x, y); // Move to the initial coordinate point (without drawing)
this.ctx.lineTo(Math.floor(x), Math.floor(y)); // Describe a line from the last coordinate point to (x, y)
this.ctx.stroke(); // Start drawing
Copy the codeSet the padding value from the horizontal axis to the edge to 30. The Canvas origin [0,0] is the top left corner of the drawing area. Here, take drawing the vertical axis of the line chart as an example. The vertical axis represents the CPU usage of 0%-100% or the memory usage of 0-1GB. We can divide the vertical axis into 100 basic units, but the coordinate points on the vertical axis can be set to 10 instead of 100 for easy viewing. So each coordinate can be represented as [0, (height-padding) -((height-(2*padding))/index) * 100], and index equals 0,10,20,30… 90, where (height-padding) is the bottom position, and (height-padding -(2*padding) is the length of the entire vertical axis.
VI. Existing known problems
1. ChildProcessPool does not work as expected in production
ChildProcessPool does not work as expected if the app is installed in the system directory in the Electron production environment. Install the app into the user directory or place the script used by the process pool to create the child process (specified by the path parameter) separately into the Electron user data directory (~/.config/[Appname] on Ubuntu20.04).
2. The UI does not listen to Console data of the main process
The main process does not support this feature. A solution is being sought.
VII. Next To Do
- Have the Service automatically restart when the support code is updated
- Add the ChildProcessPool scheduling logic
- Optimized the output of ChildProcessPool multi-process console
- Added a visual process management page
- Enhanced the ChildProcessPool process pool
- Enhanced ProcessHost transaction center functionality
VIII. Some practical examples
-
Electronux – an Electron project, I use the BrowserService/MessageChannel, and comes with a ChildProcessPool/ProcessHost using the demo.
-
File-slice-upload – a demo for parallel uploads of multiple file fragments, using ChildProcessPool and ProcessHost, based on [email protected].
-
Also look at the test sample file in the test directory, which contains the complete details for use.