JavaScript was originally designed for browsers and has no mechanism for reading or manipulating binary data streams. The introduction of the Buffer class gives NodeJS the ability to manipulate file streams or network binary streams.

Basic Concepts of Buffer

Buffer objects are allocated in C++, not in V8’s heap, but in C++. The allocation of memory is done at the Node level, and the release is automatically controlled by V8’s gc mechanism. The basic operation of Buffer is not described here. The official documentation is very detailed.

Buffer performance comparison

Generally, data needs to be converted into buffers during network transmission. Let’s do a performance comparison experiment.

1. Return a string to the client

const http = require('http');

let hello = ' '
for (var i = 0; i < 10240; i++) {
  hello += "a"; } the console. The log (` Hello:${hello.length}`)
// hello = Buffer.from(hello);

http.createServer((req, res) => {
  res.writeHead(200);
  res.end(hello);
}).listen(8001);
Copy the code

Use the ab -c 200 -t 100 http://127.0.0.1:8001/ command to perform the performance test and initiate 200 concurrent clients

Using strings, QPS can reach 4019.70 with a transfer rate of 40491.45KB per second.

2. Use buffers. The string is converted into a Buffer object and sent to the client.

const http = require('http');

let hello = ' '
for (var i = 0; i < 10240; i++) {
  hello += "a"; } the console. The log (` Hello:${hello.length}`)
hello = Buffer.from(hello);

http.createServer((req, res) => {
  res.writeHead(200);
  res.end(hello);
}).listen(8001);
Copy the code

Uncomment Buffer conversion, again using ab-C 200 -t 100 http://127.0.0.1:8001/ test, again initiating 200 concurrent clients.

Using Buffer, the QPS reached 7130.05 and the transfer rate was 71822.74KB per second. The performance is 177% of the original, greatly saving server resources. The above comparison example is from Node JS.

So why the big performance boost?

In NodeJS, if a string is returned during HTTP transmission, the string parameters will be converted into buffers, which will be returned to the client bit by bit via the Stream in NodeJS. If we return Buffer directly, there is no conversion operation and we return directly, reducing CPU reuse. See the Node source code github.com/nodejs/node…

In the performance comparison example above, when a string is returned, the string is returned as a Buffer on each request. However, when the Buffer is returned directly, the Buffer is stored in memory when we start the service and can be returned directly in memory for each request. Therefore, QPS has been improved a lot before and after the use of Buffer.

Therefore, when we write business code, some resources can be pre-converted into Buffer types (such as STATIC resource files such as JS and CSS) and directly returned to the client Buffer. For example, in some file forwarding scenarios, the obtained content can be stored as Buffer and directly forwarded to avoid additional conversion operations.

References:

  • Nodejs. Cn/API/buffer….
  • Book.douban.com/subject/257…