preface

This article is some experience gained from the engineering practice of the students in the team. Although ES6 module provides the ability to decouple code and reduce software complexity, the engineering method matching with it has always been quite troublesome. Only relying on NPM and the link capability of NPM can not adapt to large-scale modular system, so we have done some practices and attempts in this regard, and the effect is good so far. The author’s perspective is based on the needs of the team, and the writing style is more down-to-earth, so readers can read by themselves.

Author: Tax friends Experience technology team – Jiang Hao

The background,

Last month, we made a preliminary attempt to separate [client method call] and [public menu logic] of intelligent fiscal line into NPM package for unified management with the help of sub-generation access. The technical scheme of stripping public technology adopted at that time can be simply divided into the following points:

  • Each common technology package exists as a separate package.
  • Each package shares a common technology Git repository (Scavenger).
  • Each package publishes NPM packages independently.
  • Each package manages its own version.
  • Various business groups cooperate in development and release.

But in the actual development process, it is found that there are some problems in each link, as if in the primitive society, norms, workflow, production lines are incomplete.

Second, the question is replayed

2.1, internal

  • The development of collaborative
    • Branch management, pull and merge specifications.
    • Synchronization is difficult.
    • .
  • Internal dependency management
    • Internal dependencies handle the hassle of manually updating the version.
    • Different branch versions are different and lack the ability to manage dependencies uniformly.
    • .
  • Version management
    • Manually updating versions is inefficient, often forgotten and error-prone.
    • Lack of unified version management ability.
    • Version upgrade synchronization, multiple branches are prone to error.
    • .
  • Release management
    • There is no convergence of publishing authority and publishing branch, and it is easy to make mistakes and the results cover each other.
    • .
  • The internal alignment
    • Internal coordination is complex.
    • Manual link is inefficient, and link is basically unfeasible after package interdependence.
    • .
  • Production line missing

2.2, external

  • The external alignment
    • External syndication is extremely inconvenient, resulting in a proliferation of versions.
    • .
  • CHANGELOG management
    • If the version is missing, the communication cost increases, and the external cannot perceive the modification point after the version is increased.

Iii. Production line

3.1. Analogy to factory production line

In order to form an abstract concept first, we will abstract the NPM package production line to the factory production line to help memorize and understand.

The NPM package production line is similar, with the same create -> develop -> test -> package build -> release process. Our initial goal is to create a complete production line for NPM multiple packages.

3.2, according to monorepo

The concept of monorepo is, in plain English, a way of managing multiple NPM packages in a code repository (as opposed to multirepo, which means multiple packages, multiple repositories).

Many popular open source projects are using Monorepo to manage code, such as Babel and Vue3.0

  • What do we need?

First of all, it is clear that one direction of our engineering governance must be to continuously peel off public technologies and divide them into more fine-grained modules to support more flexible business and reuse. In this way, the number of NPM packages will increase rapidly and their interdependence will become more complex. In view of the increasing number of NPM packages, we urgently need a package management scheme that is perfect, coordinated, convenient for collaboration and covers the whole process as far as possible. That’s what we need.

And the way of management of many warehouses is naturally troublesome. For both collaborative and independent development, the problem complexity of multiple warehouses is bound to increase exponentially.

  • What does Monorepo solve?

Monorepo is designed to solve these problems.

“Any problem in computer science can be solved by anther layer of indirection.” – IT proverb

By integrating all packages into packages, the dependency, construction, development/debugging, testing, version and release of each package can be managed uniformly at the upper level to provide a more elegant multi-package management and collaboration scheme.

4. Details of the scheme

The goal of the production line preliminary plan is to improve and land.

Lerna and YARN Workspaces are common monorePO management tools in the market. Because lerna and YARN Workspaces overlap in many functions, The plan here is to use the recommendations provided on the YARN website (Yarn WorkSpaces aims to provide a better dependency solution and does not provide complex management functions like Lerna)

  • Use YARN Workspaces to deal solely with dependencies.
  • Use LERNA to handle the problem of managing packages collectively.

Similarly, there will be some problems that cannot be solved by tools in the initial stage. Here, we consider [specification/constraint + review] to ensure that.

4.1. Basic Directory structure

├─ └ ─ Package. json // ├─ Package. json // Package. json │ ├─ ITR-menu // Package │ ├─ readme.md │ ├─ itR-menu // Package │ ├─ itR-menu // Package │ ├─ readme.md │ ├─ ItR-menu // Package │ ├─ itR-menu // Package │ ├─ itR-menu // Package │ ├─ readme.md │ ├─ ItR-menu // Package │ ├─ itR-menu // Package │ ├─ itR-menu // Package │ ├─ itR-menu // Package │ ├─ ├─ dist │ │ ├─ lib │ │ ├─ Package. json // Package A package.json │ ├─ ├─ ├─ ├─ ├─ ├─ └.js │ ├─ └ Json │ ├─ SRC ├─ map.lockCopy the code

4.2 environment construction

The environment setup here mainly refers to lerNA installation, you can choose to install lerNA globally:

npm i lerna -g
Copy the code

Alternatively, you can use lerNA in your project, CD to install dependencies into the total REPO.

If the main project uses LERNA for the first time, it needs to initialize the main project with LERNA:

lerna init
Copy the code

Json and package.json files are generated in the root directory of the project to describe lerna dependencies and configurations. Common configuration items will be introduced in detail next.

Official documentation – lerna.json configuration item

4.3 create a new package

Using Lerna to create a new package is very convenient, just in the main project root directory, through the command line:

lerna create package-name
Copy the code

Then enter the package description as prompted to complete the creation.

Subsequent use can be considered in combination with scaffolding

4.4 Dependency management

According to the content of the previous recheck, in fact, for dependency management, we face the following problems:

  1. Subpackages dependency package volume: As the number of packages increases, the package volume increases exponentially and the dependency installation time increases exponentially.
  2. Interdependencies between sub-packages: With the increase in the number of packages, manual link and version operations in the process of development tuning are a huge burden.
  3. Dependency operations of subpackages are complex: As the number of packages increases, dependency installation and cleaning operations are complex.

Therefore, we urgently need a tool to co-ordinate and automate the management of dependencies, to solve the above problems. Yarn Workspace has the following capabilities:

  1. Yarn Workspace uses dependency analysis and dependency promotion algorithms to promote the common and common dependency packages of each package to the main project, thereby reducing the package size and installation time.
  2. Yarn Workspace analyzes the internal dependencies between packages based on the topology sequence and automatically resolves the internal dependencies in a soft link mode.
  3. Yarn Workspace allows you to install, clear, and view dependencies in a unified manner using a few simple commands under the main project.

Lerna itself has the ability of dependency management, but YARN workspaces has a more excellent algorithm of dependency analysis and dependency promotion (reactive power) (Brief Analysis of LERNA and YARN Workspace Algorithm). So here we choose to use YARN Workspaces to manage our dependencies.

4.4.1 Specific Usage and common commands

  • You need to enable YARN Workspaces in the main project. It is very simple to enable yarn workspaces in the package.json file of the main project:
{
  "name": "root"."private": true.// This is the main project and will not be published
  "devDependencies": {
    "lerna": "^ 3.22.1"
  },
  "workspaces": [              // Enable workspaces and declare subpackages (you can specify the package manually or wildcard to specify the package path)
    "packages/*"]}Copy the code

After lerNA is used to initialize the main project, YARN Workspaces are automatically enabled

Once simply enabled, the subpackage can start managing dependencies through the Yarn Workspace without making any changes.

  • Install dependencies

To install dependencies, run yarn Install (or directly yarn) in the root directory of the main project. Yarn workspace automatically handles dependency analysis, promotion, installation, and soft links.

yarn install
Copy the code
  • New | delete dependent

New | delete dependent generally divided into three scenarios:

  1. Add dependencies to a package
// Added - Third-party YARN workspace package-name add react // Added - Internal YARN workspace packageA add packageB // Delete yarn workspace package-name remove reactCopy the code
  1. Add dependencies to all packages
// Add yarn workspaces add react // Delete yarn workspaces remove reactCopy the code
  1. Added dependencies to the main project
Add yarn add -w -d react // Delete yarn remove -w -d reactCopy the code
  • Look at internal dependencies
yarn workspaces info
Copy the code

Output the dependency information of each package in the current workspaces in a tree structure (yarn version requires 1.13 or above), for example:

{
  "@itr/itr-menu": {
    "location": "packages/itr-menu"."workspaceDependencies": [
      "scavenger-client"]."mismatchedWorkspaceDependencies": []},"scavenger-client": {
    "location": "packages/scavenger-client"."workspaceDependencies": []."mismatchedWorkspaceDependencies": []},"test-package": {
    "location": "packages/test-package"."workspaceDependencies": []."mismatchedWorkspaceDependencies": []}}Copy the code

4.5 Project construction

Subpackages will have their own build commands, which can be unified according to team specifications. Lerna can be used to build all subpackages in one click. In addition, lerNA also supports building according to topological ordering rules if there are dependencies between subpackages (for example, packageB can only be built after packageA is built, otherwise it will fail). Specific commands are as follows:

lerna run --stream --sort build
Copy the code

Lerna also supports running script commands for a package individually when a standalone build is required

lerna run --scope package-name build
Copy the code

4.7 Project release

After the development and testing is over, the release phase is officially underway, but there are a few things that need to be done before the release to NPM:

version bump -> changelog -> git release -> npm publish

**

4.7.1, version bump

Version update is one of our obvious pain points before. It is not necessary to change the version number manually, and it is easy to make mistakes if there are too many packages. If there are interdependent packages, it will be a nightmare.

For these problems, LERna provides us with a great solution, which can support one-click management of version number updates of all packages, including internal interdependent packages, and one-click synchronous update. Even including push Git, build version tag, one-stop service. Lerna’s Version Bump mechanism is described in detail below.

A core command:

lerna version
Copy the code

This command will automatically do some preconditions before updating the version:

  • If there are uncommitted changes to native code, the version cannot be updated unless the changes are rolled back or committed.
  • If the code of a package has no changes compared to the code of the last release, the version update of the package without changes will not be performed.

With these lead judgments passed, the version update can be performed

lerna version --conventional-commits
Copy the code

Description Xstraw-commits parameter, which supports automatic updates of commits based on the description of the commits:

  • Feat submission exists: The minor version needs to be updated
  • Fix submission exists: Patch version needs to be updated
  • There is a BREAKING CHANGE commit: need to update a large version

After the version update is successful, Lerna will use chore(Release): publish commit commit and push it to Git to serve as the submission record of version release, and each package will automatically generate the tag of the current version and keep it for version rollback.

If you do not want to automate the description, manually specifying the version of each package without the — Straw-commits parameter is not recommended.

4.7.2, changelog

In fact, after the version command in the previous step, Lerna has automatically generated a changelog. md file for each package based on git commit record, of course, also based on [contract commit].

It is important to note that the submission instructions containing scope are used for submission, so that the submission contents of different packages will be separated into their respective CHANGELOG files when CHANGELOG is finally generated. For example:

feat(scavenger-client): xxxxx
Copy the code

4.7.3, publish,

The next step is to publish the NPM. We have already done all the preparation, just need the last line, can be based on the version generated tag NPM:

lerna publish from-git
Copy the code

It is recommended that the unified cut back to the master merge and then send the version.

4.8. Joint investigation and correlation

4.8.1, local

It is recommended to use NPM link to local results to conduct joint investigation of local codes. Specific methods:

  1. CD To the packageA directory, run the following command:
npm link
Copy the code
  1. CD to a project that depends on packageA and execute
npm link packageA
Copy the code

Then done. As simple as that, you can debug your native packageA code directly in your native project without having to release a new version.

Five, the summary

At this point, the process from package creation to release is complete, and the production line has an initial look and functionality. However, we still need to continue to improve the follow-up, including the test process, external joint investigation, get through with CI/CD, so that the process can be automated, etc. Strictly speaking, the current results are not workflow, production line (damn! Is the title party), but may lead us from “primitive society” pure manual labor transformation to “industry 1.0” steam engine instead of manual mechanical production era, the future has a long way to go, 2.0, 3.0 era more need to continue to mass, automation, modern “production line” close. With the continuous development and evolution of front-end engineering in our team, we believe that the next “industrial revolution” will come soon.

The latter

In the future, we will share more of our team’s experience and experience in engineering practice. If you are dedicated to engineering practice and are committed to thinking and solving various complex engineering problems, you may wish to pay attention to us and learn from us together.