Domestic front-end developers are more or less familiar with CNPM, but project development varies from person to person. In many cases, CNPM is not or cannot be used. Numerous projects install dependencies using YARN or other package managers. This article shows you how to speed up binary downloads in such cases.

In the recently released Github Octoverse 2019 report, JavaScript continues to be the most popular programming language. The continued prosperity of the JavaScript ecosystem is inseparable from the popularity of Node.js. When it comes to the JavaScript ecosystem, NPM is not only the package manager of choice for front-end development, but also the most important way to share code after Github. According to Snyk 2019 Open Source Security Annual Report, the NPM ecosystem has more packages than any other package manager.

At present, mainstream front-end open source projects use NPM’s online hosting service www.npmjs.com/ when they are released. However, NPM is not the only package manager that developers can use. Yarn and PNPM from the open source community are being used by more and more people. Their most notable advantage is that they speed up the installation of dependencies. CNPM, which is open source by Alibaba, is also an important option for Chinese developers.

All that CNPM stuff

Since front-end engineering development led by React and VUE has become popular in China, the dependency installation of front-end projects has become an important part of daily work. The emergence of CNPM solves the problem of slow installation speed caused by the network environment and is welcomed by a large number of domestic developers.

CNPM, which predates YARN and PNPM, uses the NPminstall module to perform installation. NPM warehouse image maintained by Taobao development team will synchronize all official MODULES of NPM regularly. CNPM will download all packages from Taobao image by default without any configuration, thus achieving the purpose of domestic acceleration. See the official document npm.taobao.org/ for details.

Although the speed of CNPM is as fast as ever, compared with other competing products, it is no longer as advantageous as when it was born. In addition, it is difficult to solve the baffling errors reported during the actual development, and there are various other factors. More and more teams are switching back to NPM, or switching to YARN or PNPM with more functionality.

Yarn is an open source package manager for the Facebook team that creates a flatter dependency tree, installs only changed modules, uses parallel downloads, and speeds up installation with native caching. PNPM, as a dark horse, has a better reputation than YARN. However, the author has no experience in using PNPM, so it will not be introduced in this article.

I think there must be some readers want to ridicule, why not use CNPM, have to toss? But there is no specific answer, every team, every person has their own situation, there is no need to criticize. It’s all about finding a problem and solving it. The next section describes how to enable YARN to obtain CNPM’s domestic acceleration capability with minimal configuration.

NPM and YARN use image acceleration

This section is nothing new; all package managers can set warehouse addresses. For details, I recommend reading the official NPM documentation.

In addition to the well-known command line configuration, you can also create.npmrc files in your project. If you commit this file to Git, you can share the configuration with all environments, making it easier for multiple people to collaborate, and it can also be used by CI and other third-party tools. Therefore, this method is recommended by the author.

# .npmrc
registry = https://registry.npm.taobao.org
Copy the code

Yarn will also read this file unless you override this configuration in.yarnrc.

# .yarnrc
registry "https://registry.npm.taobao.org"
Copy the code

Configure the mirror address of the binary file

Using domestic NPM images alone will not solve all problems. Best-known examples:

To simplify CSS writing, many projects use preprocessors. In China, one of the reasons that less preprocessor is more popular than sASS processor, which has a more complete ecosystem, is that node-Sass compilation tool used to be difficult to install, so many companies, teams and individual developers decided to use less.

What makes Node-sass difficult to install is that it triggers a separate compilation process after the NPM installation process. It uses a C++ compiled binary, which is hosted by version at github.com/sass/node-s… . Github uses Amazon’s AWS service as its CDN, which is sometimes inaccessible in mainland China for well-known reasons. As a result, when the file fails to download, native compilation is triggered to generate an alternative binary, a process that often ends in failure (especially on Windows 7). Many web tutorials at this point suggest installing a C++ related build environment, or “safe with Linux”.

But actually solving the binary download problem can greatly improve the success rate. Taobao image also provides a corresponding binary package by setting environment variables to use domestic acceleration. CNPM has this process built into it, so this situation can be resolved automatically.

If you do not use CNPM, set the environment variable from the command line:

Global configuration, permanent on a single device
yarn config set sass_binary_site https://npm.taobao.org/mirrors/node-sass/

# For single installation
SASS_BINARY_SITE=https://npm.taobao.org/mirrors/node-sass/ && yarn add node-sass
# or
yarn add node-sass --sass_binary_site https://npm.taobao.org/mirrors/node-sass/
Copy the code

Or you can also configure it in a.npmrc file to share the configuration:

# .npmrc
sass_binary_site = https://npm.taobao.org/mirrors/node-sass/
electron_mirror = https://npm.taobao.org/mirrors/electron/
Copy the code

Phantomjs and Electron can also be accelerated using this method, as they both allow the image URL to be set using environment variables. We can view all available images at npm.taobao.org/mirrors.

That’s where most of the articles on the Web end, but that’s not all.

Use the bin – wrapper – China

Imagemin is a series of image compression modules based on C++, including well-known libraries such as pngquant and mozjpeg. Like node-sass, imagemin requires downloading binaries. However, it does not exclude the use of environment variables to configure the mirror repository URL, and the success rate of self-compilation is much lower. At this point, both NPM and YARN can only hope that the network is smooth.

Yes, CNPM also solves this problem with built-in processing, so is it necessary to go back to CNPM?

Of course not, a look at the source code shows that quite a few modules that use binaries are downloaded and compiled via bin-wrapper. As a result, the problem can be solved by replacing the download link used in the bin-wrapper with the mirror repository URL before downloading.

The author created a tool called bin-wrapper-China, which forked the original bin-wrapper and read binary-mirror-config used by CNPM to get all available images taobao image urls, replacing the download file link. This makes it fun to use the acceleration feature. How to download bin-wrapper-china instead of bin-wrapper?

The answer is to use yarn’s killer function (not supported by NPM), which allows us to replace another module with the specified one when the installation is performed with YARN, as follows:

// package.json with yarn
{
  "resolutions": {
    "bin-wrapper": "npm:bin-wrapper-china"
  }
}
Copy the code

The “impersonation” of bin-wrapper-China takes place during installation, and the running of bin-wrapper takes place after installation, so it runs seamlessly. This will greatly improve the success rate of the Imagemin series. For resolutions, see:

  • Yarnpkg.com/lang/en/doc…
  • Github.com/yarnpkg/rfc…

For modules that support environment variables, such as Node-sass, bin-wrapper-china can also support china-bin-env instead of manual environment variables. However, since we do not recommend injecting YARN or NPM itself, and the injection of environment variables must be performed prior to installation, manually set the preinstall command within the project if necessary:

// package.json
{
  "scripts": {
    "preinstall": "china-bin-env || echo 'Fail: china-bin-env'"
  }
}
Copy the code

The preinstall-based operation requires a pre-installation of bin-wrapper-China, and I hope there will be a better solution later.

conclusion

We may decide to discard CNPM due to its lack of functionality, but its acceleration capability is what we need.

To sum up, CNPM does three things:

  • Use Taobao NPM image warehouse to speed up the installation of regular modules.
  • Configurable binaries that are accelerated by injecting environment variables ahead of time.
  • Unconfigurable binary file, forcibly replaced URL to speed download.

These are also the three things we do (usually configured in projects) :

  • The. NPMRC file is used to set the NPM warehouse address to the domestic mirror address.
  • Configure environment variables through the.npmrc file, or throughbin-wrapper-china 的 china-bin-envCommand to inject environment variables.
  • To configure yarn resolutions, usebin-wrapper-chinaPretend to bebin-wrapperImplement URL replacement.

Of course, if you are determined to use CNPM, or if you are working on an unblocked network, or if your project doesn’t need to install binary-containing modules (for example, I used Sass instead of Node-sass in my project), you don’t need to worry about this. Although yarn is recommended in this article, its core process applies to most package managers in the Node.js ecosystem. You can explore more if you are interested.

Related projects:

  • Github.com/best-shot/b…
  • Github.com/cnpm/binary…