Why would you let these two bump into each other?

As we all know, node.js is sometimes used to perform complex computations that block the main thread and slow down the response time. We all know that Node.js is suitable for I/O intensive applications and not CPU intensive applications for this reason. So how to solve this bottleneck? Node.js has the ability to call native modules, such as C/C++ related modules, but our main topic today is not C/C++ but Rust, Rust can also use FFI(External Function Interface) abstraction to create native modules that can be called by C/C++/Python/Node.js, and so on. To invoke the Rust module in Node.js, we need neon’s help.

Understand the neon

Neon (link), which contains a set of tools and glue code to help Node.js developers become more productive, allowing them to write native Node.js modules in Rust and integrate them seamlessly into JavaScript code. Using Neon makes it possible to create a native Node module just like in C/ C++, and it’s easy to use without the fear or headache of having to live in unsafe code (one of Rust’s features is writing secure code). Writing native modules in Rust has the following benefits:

  • Provides native performance
  • Can write multithreaded concurrent programs
  • You can useRustCommunity ecology, various open source packages
  • Can call the local operating systemAPI

Application scenarios

Native Addon can do some things that JavaScript can’t, such as call system libraries, open a window, call gpus and other system instructions, and in some CPU intensive computing places, such as blockchain currency calculations, File encryption decryption and so on need relatively high computing power, at this time we can use the native module to expand, the essence of the native module is a binary file. What are the well-known application scenarios of the front-end?

Such as:

  • swc(A link to the) :swcIs a use ofRustWritten super super fastTypescript / JavascriptCompiler (similar tobabel), it is one that can be given simultaneouslyRust 和 JavascriptThe library used. Want to inRust, can refer to hereA link to theAnd you want toJavascript, can refer to hereA link to the.
  • Next.js (> v12.0): the latestnext.jsIs also used inRustNative modules built as compilers,next.jsThe compiler is based on the aboveswcAccording to the official introduction, local recompile speed after using the latest compilerThat’s more than a threefold increase, packaging speed in production environmentAn increase of more than five timesAnd compile speedMore than 17 times faster than Babel.A link to theHere you are.
  • There are also some third-party open source usesRustWrite thenpmPackages, such as encryption algorithmsnode-rs/bcryptChinese participlenode-rs/jiebaAnd so on, which all involve complex calculations.

Begin to understand and implement requirements

We now want to use Rust to write a function that counts the number of occurrences of a given word in a chunk of text, and make it available to the Node.js side.

Make sure you have Rust and Node.js(version > 10.0) installed on your computer before you use the following command to create a development folder:

npm init neon my-project
Copy the code

Next, enter the relevant project information as prompted, press Enter to skip all of it, and subsequent modifications can be made in package.json and Cargo. Toml.

After the command is created, we can see a directory structure like this:

└ ─ native_counter │. Gitignore │ Cargo. Toml │ package. The json │ README. Md └ ─ SRC lib. RsCopy the code

In the SRC directory, we can see that lib.rs is generated for us by default, and that’s where we write the code. Let’s take a look at the Cargo. Toml file that was generated for us by default.

[package]
#.. Leave that out

[lib]
crate-type = ["cdylib"]

[dependencies]

[dependencies.neon]
version = "0.9"
default-features = false
features = ["napi-6"]

Copy the code

Let’s look at the crate-type field, which means:

[crate_type = “cdylib”] – A dynamic system library will be generated, similar to the C shared library. This property is used when compiling a dynamic library that loads calls from other languages. On Linux, files of type *.so are generated, on MacOS files of type *.dylib are generated, and on Windows files of type *.dll are generated.

The following [dependencies. Neon] provides information about NEON’s dependencies

Write the code

Open the file native/ SRC /lib.rs and start writing our code:

// Import all neon attributes
use neon::prelude::*;

Cx contains some context information about the function. JsResult is a generic return type
fn count_words(mut cx: FunctionContext) -> JsResult<JsNumber> {
    // The first argument to the JS function, of type string
    let text = cx.argument::<JsString>(0)? .value(&mut cx);
    // The second argument to the JS function, of type string
    let word = cx.argument::<JsString>(1)? .value(&mut cx);
    // Return a number, count the number of words in the text, lower case all words, then split by space, filter to count the number
    Ok(cx.number(
        text.to_lowercase()
            .split("")
            .filter(|s| s == &word)
            .count() as f64,))}#[neon::main]
fn main(mut cx: ModuleContext) -> NeonResult<()> {
    cx.export_function("count_words", count_words)? ;Ok(())}Copy the code

The FunctionContext instance is a parameter to the function, which contains information about the function being called in JavaScript, such as the parameter list, parameter length, this binding, and other details.

After writing the Rust code, the next step is to generate the Node module from the project and run the command in the root directory of the project:

npm install
Copy the code

The dependencies will then be automatically installed and the corresponding cargo compilation commands will be executed. When you’re done, you’ll see a new file named index.node generated in the root directory. This file is the Rust module code we just wrote. Before writing our test code, let’s take a look at the file types of node.js loading modules

Node.js loads the file

Node.js require supports three main file types:

The.js:.js file is the most common type of file we use. It loads the entire js file and returns the module.exports as require.

Json: the. Json file is a plain text file that can be converted directly to an object using json.parse.

.node:.node files are compiled binaries in other languages.

If no extension is specified, for example const test = require(‘./index’); , node.js will try to parse it as a.js file first, and if it is not a.js file, it will try to parse it as a.json file. If neither, parsing is attempted as a.node binary.

test

We create a test.js file in the project root directory to test and write the relevant code

const rust_test = require('./index.node')

let testText = 'Nice to meet you'

console.log(rust_test.count_words(testText, "you")) / / output 1
Copy the code

This is an example of how to write a simple Node.js call to a native module written by Rust. If you want to write a more complex example, you need to have a better foundation of Rust.

Other related introductions

Since the release of Node.js V10, Node.js has introduced an improved interface for developing native modules, n-API (related links), which is the principle behind neon’s construction here. Node.js exposes the APPLICATION Binary Interface (ABI) to native Addon developers.

The majority of Node.js addon in the community is developed in C/C++. The C/C++ ecosystem is so thriving that you can find C/C++ libraries for almost anything you want to do. However, C/C++ lacks a unified build tool chain and package management tools, resulting in many difficulties in development and maintenance. Rust, on the other hand, has the modern management tool Cargo (good and powerful).

Since writing native modules in Neon is so powerful, isn’t there a downside to it?

That’s certainly true. Native code (C/C++/Rust) is much faster than pure JS in some pure computing scenarios, but once you use n-API to work with Node’s JS engine, there is a significant performance overhead (relative to computing). In some simple calculation operations, using native modules for development may be slower than pure JS, because in the process of bridging the performance cost has exceeded the time of its own calculation, the result is not worth the loss, so why recommended in complex calculation scenarios, For example, compiling and packaging files in SWC is one of its application scenarios, which is complex enough. So our choice depends on the application scenario.