Monorepo of actual combat

preface

Because many online articles about Monorepo share with do will have a lot of error, so want to record…

It was a headache to maintain the common code of multiple warehouses. Every change to the common code had to be synchronized with the full warehouse. Finally, I decided to use Monorepo to transform it.

Monorepo

Monorepo(monolithic repository) is a monolithic way of managing project code. It refers to managing multiple modules/packages in a single project repository (REPO), rather than the common practice of building a single REPO for each module.

Many large open source projects have adopted this approach, such as Babel, React, Vue, etc. The Monorepo management code can manage (build, test, publish) multiple packages with just one set of scaffolding.

The contents of the first level directory of the project are mainly scaffolding. The main contents are managed in packages directory and divided into multiple packages. The directory structure is roughly as follows:

├ ─ ─ packages | ├ ─ ─ two packages pkg1 | | ├ ─ ─ package. The json | ├ ─ ─ pkg2 | | ├ ─ ─ package. The json ├ ─ ─ package. The jsonCopy the code

One problem here is that while it is much easier to disassemble the NPM package to manage the project, debugging becomes difficult when the warehouse contents are related. So the ideal development environment would be one that only cares about business code, that can be reused directly across businesses regardless of how, and that all code is in the source code when debugging.

The most common Monorepo solution today is the workspaces feature of LERna and YARN. Yarn handles dependencies and LERNA handles publishing.

Lerna

Lerna is a management tool for NPM modules, providing projects with a directory mode for centralized package management, such as unified REPO dependency installation, Package scripts, and release features.

The installation

Recommended global installation

npm i -g lerna
Copy the code

Initialize the project

lerna init
Copy the code

After initialization, empty packages directories and package.json and lerna.json configuration files are generated as follows:

//package.json {"name": "root", "private": true, // "devDependencies": {"lerna": "^ 3.22.1"}} / / lerna. Json {" packages ": [" packages / *"], "version" : "0.0.0"}Copy the code

Create NPM package

You can modify the package information by running the command. Here create @monorepo/ Components and @monorepo/utils

lerna create @monorepo/components
Copy the code

Install dependencies

Lerna and Yarn Workspace install dependent methods are useless

Lerna add lodash // add lodash module to @monorepo/utils As @monorepo/components) lerna add lodash --scope @monorepo/utilsCopy the code

The chicken side of Lerna Add is that you can only install one package at a time…

Dependency package Management

Generally, package dependencies are in their own node_modules directory, which not only increases the installation and management costs of the package, but also may lead to multiple dependencies. So all package dependencies can be promoted to the project root directory.

Both Lerna and Yarn Workspace can promote dependent packages to the REPO root for management. Lerna provides the –hoist option when installing dependencies, but the nifty part is that because Lerna compares dependency versions with strings, dependencies are promoted to the root directory only when the versions are exactly the same. For example:

A depends on @ Babel /core@^7.10.0 and B depends on @ Babel /core@^7.11.4Copy the code

Lerna will install 7.11.4 Babel /core in node_modules of A and generate A package-lock.json in A, which undoubtedly increases the maintenance cost and package size.

In this case, the Yarn workspace will only have a copy of yarn-lock.json in the root directory and will not repeatedly install dependencies in subdirectories.

yarn workspace

Set up the environment

This is mainly for configuration adjustments for installation dependencies.

In the project managed by Monorepo, there are dependencies between libraries, such as A depends on B, so we usually need to link B to NODE_module of A. Once there are many warehouses, manual management of these link operations is A great burden, so automatic link operations are needed. Link dependencies in topological order

Solution: Use workspace, yarn Install will automatically help solve the installation and link problems

Yarn install # is equivalent to lerna bootstrap --npm-client YARN --use-workspacesCopy the code

Package. json & lerna.json is as follows:

Json {"packages": ["packages/*"], "npmClient": "yarn", "useWorkspaces": true, // Use yarn workspaces "version": Json {"name": "root", "private": true, "workspaces": [// specify workspace path "packages/*"], "devDependencies": {"lerna": "^3.22.1"}}Copy the code

Clean up the environment

Clean up dependencies in the case of broken dependencies or messy projects

Yarn workspaces run clean # Clean all packages node_modules directoryCopy the code

Lerna Clean can remove root dependencies

lerna clean does not remove modules from the root node_modules directory, even if you have the –hoist option enabled.

Install/remove dependencies

There are three scenarios

Install/remove dependencies for a package
Install/remove dependencies from root. Common development tools such as typescript are installed in root
Install/remove dependencies for all packages

– error Invalid subcommand. Try “info, run”, Yarn 2.0 can only use YARN workspaces foreach. You need to install workspace tools before running yarn 2.0. So I’m only going to say the first two cases.

The dependency installation/deletion scenarios are as follows:

Yarn workspace packageA add/remove packageB [packageC -d -w] // Install and delete packageB and C dependencies yarn add/remove typescript for packageA -w -d install/delete typescript for rootCopy the code

⚠️ Note: If yarn workspace packageA add XXX is used, all packageA dependencies will be installed again and installed in the packageA directory. To install to the root directory add configuration -w

For installing local Dependency, the implementation of YARN is temporarily buggy. You need to specify the version number for the first installation, otherwise the installation will fail as follows:

If uI-button is not published to NPM, yarn workspace UI-form add ui-button will fail to install, but yarn workspace UI-form add [email protected] will succeed

The dependency file structure is as follows:

Submit specifications

There is also some configuration that needs to be done about code submission before building and publishing

commitizen && cz-lerna-changelog

Commitizen is a tool for formatting Git commit messages and provides an interrogative way to get the required commit information.

Cz-lerna-changelog is a submission specification specially tailored for LERNA project. In the process of inquiry, the selection of packages will be similarly affected. As follows:We use Commitizen and CZ-Lerna-Changelog to standardize the submission in preparation for automatic log generation later. Since this is a development dependency for the entire project, install it in the root directory:

yarn add commitizen cz-lerna-changelog -D -W
Copy the code

After installation, add config field to package.json to configure CZ-Lerna-Changelog to Commitizen. Also, since Commitizen is not globally secure, scripts need to be added to perform git-cz

{ "name": "monorepo", "private": true, "workspaces": [ "packages/*" ], "scripts": { "commit": "git-cz" }, "config": { "commitizen": { "path": "./node_modules/cz-lerna-changelog" } }, "devDependencies": { "commitizen": "^ 2", "cz - lerna - changelog" : "^ 2.0.3", "lerna" : "^ 3.22.1"}}Copy the code

You can then use YARN Run Commit in normal development to input the code step by step as prompted.

commitlint && husky

The following configuration forces developers to follow the above specification, but can be skipped because it takes a while to commit…

We used Commitizen above to regulate commit, but it is difficult to develop yarn Run Commit on your own. What if you forget, or commit directly with Git? Therefore, verify the submission information at the time of submission. If it does not meet the requirements, do not allow the submission and prompt. Validation is done by commitlint, and the timing of validation is specified by HusKY. Husky inherits all of Git’s hooks, preventing illegal commits, pushes, etc.

Install CommitLint and specifications to follow:

yarn add -D -W husky @commitlint/cli @commitlint/config-conventional
Copy the code

Add the configuration file commitlint.config.js to the project root directory to specify the appropriate specification for commitLint

module.exports = { 
	extends: ['@commitlint/config-conventional'] 
}
Copy the code

Add the following configuration to package.json

"husky": { 
 		"hooks": { 
    		"commit-msg": "commitlint -E HUSKY_GIT_PARAMS" 
     }
 }
Copy the code

“Commit-msg” is a hook that validates commit information on git commit. When triggered, commitLint is used for validation. After the installation and configuration is complete, if you want to commit through git Commit or other third-party tools, you cannot commit as long as the submitted information does not meet the specifications. Bind the developer to commit using YARN Run Commit.

eslint && lint-staged

Originally intended to skip the ESLint specification first… However, if the project already has ESLint and the NPM package does not, debugging will return an error. You would think that a.eslintlrc file and installing the ESLint plugin would solve the problem. Later, the problem solving process is complemented

In addition to the specification submission information, the code itself is certainly less dependent on the specification to unify the style.

yarn add -D -W standard lint-staged
Copy the code

Eslint is a complete set of JavaScript code specifications with automatic linter & code correction. Automatic formatting and correction of code, early detection of style and application problems, as well as javascript code specification verification, eslintrc.json:

module.exports = {
  env: {
    browser: true,
    es2020: true
  },
  extends: ["eslint:recommended", "plugin:vue/essential"],
  parserOptions: {
    parser: "babel-eslint"
  },
  plugins: ["vue"],
  rules: {
    "prettier/prettier": [
      "off",
      {
        quotes: 0
      }
    ]
  }
}
Copy the code

Lint-staged is a Git term for staging areas, where only files are checked and corrected. On the one hand, it can improve the efficiency of calibration, and on the other hand, it can bring great convenience to old projects.

Package. The json configuration

{ "name": "monorepo", "private": true, "workspaces": [ "packages/*" ], "scripts": { "c": "git-cz" }, "config": { "commitizen": { "path": "./node_modules/cz-lerna-changelog" } }, "husky": { "hooks": { "pre-commit": "lint-staged" } }, "lint-staged": { "*.(vue|js)": [ //"eslint --fix", "prettier --write" ] }, "devDependencies": {" commitizen ":" ^ 2 ", "cz - lerna - changelog" : "^ 2.0.3", "lerna" : "^ 3.22.1", "lint - staged" : "^ 10.2.13", "standard" : "^ 14.3.4}}"Copy the code

After installation, lint-staged configuration “prettier –write” was added to package.json, validation time was set to pre-commit, and pre-commit hooks were added to husky configuration to perform Lint validation.

Eslint –fix verify and automatically fix

Build and publish using Lerna

The project build

Each package is interdependent. For example, packageB can only be built after packageA is built, otherwise errors will occur, which actually requires us to build according to a topological ordering rule.

We can build our own topological sorting rules. Unfortunately, yarn workspace does not support executing commands based on topological sorting rules. Although the RFC has been accepted, it is not implemented yetFortunately, Lerna supports executing commands by topology sort, and the –sort parameter controls executing commands by topology sort

lerna run --stream --sort build
Copy the code

This can be configured under package.json in the root directory

"scripts": {
    "build": "lerna run --stream --sort build"
  },
Copy the code

Version upgrade and package delivery

After so many difficulties, we finally arrived at the final step of release

After project testing is complete, release is involved, and release usually involves the following steps

Conditional validation: verify that tests pass, that there is uncommitted code, and that there is a release on the main branch
Version_bump: The version number needs to be updated when releasing a version. How to update the version number is a problem, usually following semVer semantics
Generate Changelog: In order to view the functions solved by each version of each package, we need to generate a Changelog for each package so that users can view the function changes of each version.
Git Tag generation: Create a Git tag for each version to facilitate subsequent rollback problems and troubleshooting
Git releases: For each release we need to generate a separate commit record to mark milestone
Release the NPM package: After releasing Git, we need to release the updated version to NPM for external users

Yarn does not intend to support the release process. It only wants to be a package management tool, so LerNA is required to support this part

Lerna provides publish and version to support version upgrade and release. Publish can include version or publish only.

Only publish a package

Lerna official does not support releasing only one package, github.com/lerna/lerna…

Lerna will automatically monitor whether the git commit record contains the file modification record of the specified package to determine the version update, which requires setting a reasonable ignore rule (otherwise it will cause frequent and meaningless version updates). The advantage is that it can automatically help update versions between packages

For example, if uI-Form relies on UI-button, the uI-Button version dependency will be automatically updated to the latest version of UI-Button if the VERSION of UI-Button changes. If the UI-Form version changes, the UI-Button will not be affected.

As tested, version_bump relies on file detection combined with subject, not scope, which is used to generate Changelog. If uI-form files are modified, While the commit record writes fix(UI-button), lerna generates uI-Form version updates, not uI-button version updates.

Publishing automatically generates logs

With the previous specification submission, automatic logging came naturally. Leran publish does the following:

Lerna Version Updated version

Find the packages that have changed since the last release
Prompts developers to determine the release number
Update the version field of package.json in all updated packages
Update the dependency version numbers in packages that depend on the updated package
Update the version field in lerna.json
Commit the above changes with a tag
Push to git repository

lerna publish

Description The parameter “Xstraw-commits” can be used to automatically identify updates based on the Conventional COMMIT specification and git Commit message.

// lerna.json { "packages": ["packages/*"], "npmClient": "yarn", "useWorkspaces": true, "command": { "version": {"conventionalCommits": true # Generated changelog files and changed versions according to commit}}, "ignoreChanges": [" / *. * * md "], # md file update, don't trigger the change version "version" : "0.0.0"}Copy the code

Package. json within the package also requires the publishConfig configuration

"PublishConfig ": {"access": "publish" // If this module needs to publish, for scope module, publish, otherwise permission validation is required}Copy the code

Finally, the command is issued

lerna publish [from-git]
Copy the code

If the first attempt failed and the result was successful, you will need to add from-git to the file.

Complete test cases

Monorepo Project: There are two ways to test

The advantage of using a unified jEST test configuration is that it is convenient to run jEST globally. The disadvantage is that if the package is heterogeneous (such as applets, front-end, Node server, etc.), the unified test configuration is not easy to write
Each package supports the test command separately, and yarn workspace run test is used. The disadvantage is that the test coverage of all codes is not collected uniformly

Here is a sample typescript test that initializes the jest.config.js configuration:

module.exports = {
  preset: 'ts-jest',
  moduleFileExtensions: ['ts'],
  testEnvironment: 'node'
}
Copy the code

Finally, attach github address: github.com/moon-bonny/…

The READme is not ready yet…

Vue component package

To be continued…

There will be time to step on the pit

Part this article reprinted in: mp.weixin.qq.com/s/NlOn7er0i…

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

preface

Monorepo

Lerna

The installation

Initialize the project

Create NPM package

Install dependencies

Dependency package Management

yarn workspace

Set up the environment

Clean up the environment

Install/remove dependencies

Submit specifications

commitizen && cz-lerna-changelog

commitlint && husky

eslint && lint-staged

Build and publish using Lerna

The project build

Version upgrade and package delivery

Only publish a package

Publishing automatically generates logs

Complete test cases

Related articles

Monorepo of actual combat

preface

Monorepo

Lerna

The installation

Initialize the project

Create NPM package

Install dependencies

Dependency package Management

yarn workspace

Set up the environment

Clean up the environment

Install/remove dependencies

Submit specifications

commitizen && cz-lerna-changelog

commitlint && husky

eslint && lint-staged

Build and publish using Lerna

The project build

Version upgrade and package delivery

Only publish a package

Publishing automatically generates logs

Complete test cases

Related articles

Related Posts

Javascript function implementation topics

Understand the cycle of events

Browser cross-domain access-Control-allow-headers problem