This article aims to document all the technical points about Monorepo and best practices for managing multiple Node package development based on typescript + YARN Workspaces + LERNA.

NOTE: This article describes the scenarios developed by library. The practices may not meet all monorePO scenarios. The article will not comprehensively introduce various scenarios of MonorePO application, but some simple explanations will be given when necessary in some chapters.

Difference between NPM and YARN

As we know, NPM and YARN are both native Node Packge management tools. They have mutually compatible functions in package management, such as dependency Management, publish, install, and so on.

NPM is released with the release of Node and is a package management tool maintained by Node.

Yarn is a package management tool independently developed and maintained by Facebook to improve the dependency installation performance of NPM in large-scale projects. In terms of dependency management, Yarn greatly improves the installation performance in the following aspects: The dependency consistency in distributed distribution system is guaranteed (yarn.lock), which solves the problem of inconsistent application performance caused by inconsistent dependency versions in distributed distribution system.

Since NPM5, NPM has gradually filled in the performance and dependency consistency gaps, so it is possible to use YARN or NPM as a standalone dependency management project.

The biggest difference between YARN and NPM is that yarn native introduces the workspaces concept, which enables projects using YARN for dependency management to have the native mono-repo capability, while NPM native has no equivalent functionality. The ability to implement workspaces without yarn had to be implemented with LERNA. In the following chapters, we will introduce mono-repo & YARN WorkSpaces & Lerna in detail.

What is a Mono – repo

A concept corresponding to mono-repo, multi-repo. These two concepts are really different solutions to the same problem.

In the process of multi-project development, there will always be some reusable logic. At this time, we will package several reusable logic into the same package through the correlation of reusable logic, and then we may also expect that according to the correlation of package, When several packages are aggregated together for management (version control, git/SVN), two different management schemes emerge:

  • Multi-repo: In the beginning, many developers will create a Git repository for each independent package. Multiple packages will have multiple independent Git repositories, which are called multi-repo.

  • Mono-repo (multi-pack warehouse) : The biggest problem with multi-repo is that when the same developer maintains multiple packages at the same time and there are dependencies among these packages, it will be quite troublesome to manage. It not only needs to check out multiple repositories, but also needs to manually update its dependencies after a package is updated. In addition, the same dependency of different packages requires multiple copies to be installed. Therefore, in order to solve the above problems and improve development efficiency, Mono-repo was born. Mono-repo manages a set of related packages in the same Git repository. Co-dependencies between different packages are enhanced, and dependencies between packages are automatically updated when one of them is updated (or not, Described in the following sections) and new releases.

Why and when is Monorepo used

So, from the introduction of the previous chapter, we learned that the core criteria for using Mono-repo are: For managing a set of packages that are related, we can try to ask ourselves the following questions to determine whether we need to use Mono-repo:

  1. Is the set of packages being maintained relevant? For example, it is a library of some tool classes, or it is a package of different parts of a series of tools (React), or it is a different plug-in system (bable).

  2. Are these packages interdependent? For example, you have a core package that is dependent on multiple other packages, and you need to upgrade the versions of the dependent parties when the core version changes

  3. Are there more co-dependencies between these packages? (With limitations, appropriate reference)

How to create Monorepo

Starting with this chapter, we need to get our hands dirty on how to create mono-REPo and integrate some best practices to make our projects more standard and efficient.

There are many schemes to create Mono-repo. Due to the author’s energy, this article only introduces two popular and active schemes yarn Workspaces and LERNA.

Let’s start with a quick comparison of these two schemes:

  • The same

    • Mono-repo can be created independently

    • Dependencies between packages (described below as Local dependency) can either be linked locally using Syslink or directly install versions that have been published to the specified Registry

    • Packages co-dependencies can be promoted to the root directory to avoid repeated installs. Lerna does not promote by default. Hoist parameter is displayed for bootstrap

  • The difference between

    • Yarn Workspaces does not provide automatic semantic version management between local dependency and unified package release. You need to manually update or publish the version in each package

    • Lerna uses NPM as the package management tool by default, but uses npmClient to change to YARN. However, if yarn is only used as the dependency management tool, there is little difference between YARN and NPM

Yarn2 has made a lot of improvements to Workspaces, and also provides the management of local Dependency version update and the release of Workspaces, etc. Those who are interested can study them in depth. The author will also organize some experiences and official documents after subsequent practice

From the above comparison, we can find that the scheme of YARN workspaces for mono-repo management is not complete. For example, using yarn workspaces is not enough for library-type mono-repo management with explicit version management like React or Babel, but for a small Web project developed by a node, This project is composed of the client side, the server side and a component library developed by ourselves. For this type of project, there is no strong burden of version management (only required by the component library), and each sub-project has certain independence. In order to improve the construction efficiency in the development scenario, Creating mono-repo using only YARN workspaces is a very lightweight option.

Therefore, the recommended scheme is the combination of YARN Workspaces and LERNA when managing the multi-package warehouse of library. Yes, the two schemes can be perfectly combined. The yarn workspaces itself is a low-level capability. Lerna itself only uses the capabilities provided by NPM or YARN to work, so we can switch larna’s underlying multi-packet capabilities to YARN workspaces while using leran’s additional commands for package versioning and publishing.

To create a

The full project can be found at github.com/dancon/mono… .

It is assumed that you have installed Node and yarn 1.x or 2.x globally. If you have not installed yarn 1.x or 2.x globally, you can refer to the following documents to install it yourself:

  • Download | Node.js

  • Installation | Yarn

  • Create a remote project on Github with the remote address

  • Check out the remote repository and initialize the project, demonstrated here using Monorepo

Check out the code base
git clone <remote-repo url>

Go to the project root directory
cd monorepo

# Initialize as NPM project
yarn init --yes
Copy the code


replace with the github repository address you created

The project has only one package.json file at this point

  • To enable YARN Workspaces, modifypackage.json, for specific referenceClassic.yarnpkg.com/en/docs/wor…
{"private": true, // Avoid root project being published "workspaces": ["packages/*"] // For now fill in as packages/* specify workspace location as packages}Copy the code
Make sure that under the monorepo directory, create the Packages directory as a YARN workspace or subproject
mkdir packages
Copy the code

At this point, the project is working in YARN Workspaces mode.

  • Configure LERNA using YARN workspace
Install lerna to the root directory using YARN add -w
yarn add -W -D lerna

Initialize lerna using independent mode
yarn lerna init --independent
Copy the code

If the -w parameter is not specified when yarn add is added, an error occurs

Create your own.gitignore file

Generate the lerna.json configuration file with the following contents:

{
  "packages": [
    "packages/*"
  ],
  "version": "independent"
}
Copy the code

Add the following configuration to have LERna switch to using YARN for dependency management and use YARN workspaces

{// This configuration is optional, some ides (VS Code support) will read this JSON schema for the configuration item intelligent prompt "$schema": "http://json.schemastore.org/lerna", "packages": [ "packages/*" ], "version": "independent", "npmClient": "Yarn ", // tell LERna to use yarn as package management tool "useWorkspaces": true // Use yarn workspaces}Copy the code

At this point, yarn workspaces + LERNA is supported for management.

  • Initialize the typescript
# install dependencies
yarn add -W -D typescript

Initialize in a directory
yarn tsc --init
Copy the code

Modify tsconfig.json in the root directory to the following content

{ "compilerOptions": { "target": "es5", "module": "commonjs", "allowJs": true, "declaration": true, "declarationMap": true, "sourceMap": true, "strict": true, "esModuleInterop": true, "forceConsistentCasingInFileNames": True}, // Exclude some files that do not need to be processed. Some of these files and folders are described in detail in the following sections: [ "node_modules", "packages/**/node_modules", "packages/**/lib", "packages/**/test", "packages/**/*.test.ts", "packages/**/jest.config.js" ] }Copy the code
  • Create packages
Create two packages, core and PKg1
cd packages
mkdir core
mkdir pkg1
Copy the code

The directory structure of each package is as follows:

.├ ─ SRC │ ├─ ├# source directory├ ─ ─ jest. Config. JsJest --init├ ─ ─ package. Json# yarn init --yes└ ─ ─ tsconfig. JsonTs configuration of package
Copy the code

The specific configuration of jest.config.js is highlighted in the following sections

Package. json focuses on the following configuration:

{ "name": "@scope/core", "main": "lib/index.js", "types": "lib/index.d.ts", "publishConfig": "Access ": "public"}, "devDependencies": {"@types/node": {"@types/node": { "^ 13.13.5"}, "scripts" : {" test ":" jest ", "build" : "the rm - rf lib && TSC" add build scripts / /}}Copy the code

Tsconfig. json configuration is as follows:

{ "extends": ".. /.. Json ", // inherit the tsconfig.json from the root directory to configure "compilerOptions": {"rootDir": "SRC ", "outDir": "lib"}}Copy the code

I will focus on project dependency management in Mono-repo, covering the following topics:

  • The installation of the package

  • Public Package Installation

  • Install packages for all packages

  • Installing local dependencies

The project is combined with Lerna + Workspaces, so the package can be installed in two ways:

# Install using yarn Workspace
yarn add -W <package-name>
yarn workspace <workspace-name> add <package-name> # 
      
        is the name in the corresponding package package.json
      

Install via LERna
lerna add <package-name> Install dependencies for all Packages /*
lerna add <package-name> --scope <workspace-name> # effect as yarn workspace
Copy the code

NOTE:

Yarn add -w effect and lerna add are not specified –scope is not equivalent

Yarn add -w is a common dependency for installation. Update package.json in the root directory

Lerna add does not specify –scope installs dependencies for all packages, updates all packages/**/package.json

If we need to add a local dependency for our package, such as @scope/ Core for @scope/pkg1, we can either install it using YARN Workspace or lerna add –scope

Yarn workspace @scope/pkg1 add @scope/[email protected]# is equivalent toLerna add @scope/[email protected] --scope @scope/pkg1Copy the code

The above method specifies the same or compatible version as the local @scope/core, so @scope/core points to the local repository via syslinks

If we do not specify the version, yarn or LERna will pull online packages from NPM Registry, and in this case, the packages in node_modules are referenced in our project

Local Dependency specifies the same or compatible version number

Not specifying the version

Version management and release

Yarn Workspaces does not provide unified version management and release. If lerNA is not used, yarn Version and YARN publish can be used to manage each other. However, dependencies between them need manual management, which is inefficient and prone to errors.

In order to use lerna’s capabilities, git message can be used for automatic version management through commit Convention, and the version of dependent parties can be automatically updated after package version changes, and each package can automatically generate Changelog.md. We just need to do the following configuration:

lerna.json

{ "packages": [ "packages/*" ], "version": "independent", "npmClient": "yarn", "useWorkspaces": true, "command": {"publish": {"conventionalCommits": true, // Enable Commit Convention "Https://registry.npmjs.org/", / / the specified registry "message" : "chore: Publish "// Commit Message prefix}, "version": {"message":" Chore: publish"}}Copy the code

Instead of automatically updating the version with a Commit Message for the first release, from-package can be used

lerna publish from-package
Copy the code

Add the following scripts command to package.json in the root directory

// add release script "build": "lerna exec --stream yarn build", "test": "lerna exec -- yarn test --passWithNoTests", "lint": "eslint --ext js,jsx,ts,tsx packages --fix", "gen": "plop" } }Copy the code

It is then executed through YARN in subsequent publications

# First release
yarn release from-package

# post
yarn release
Copy the code

For information about commit Convention, see: Conventional Commits

A separate note about the Commit message in Mono-repo is that it is a good idea to specify a scope for each package modification at each commit, such as:

git commit -m 'feat(core): add a new feature'

We can also use CommitLint to regulate our commit information, as detailed in the following sections.

yarn workspaces

The official document: yarn workspace | yarn

Let’s focus on some common commands:

  • yarn workspaces infoShow all workspace dependencies in the current project

The result is as follows:

Yarn machine-specific v1.21.1 {"@pandolajs-test/core": {
    "location": "packages/core"."workspaceDependencies": []."mismatchedWorkspaceDependencies": []},"@pandolajs-test/pkg1": {
    "location": "packages/pkg1"."workspaceDependencies": [
      "@pandolajs-test/core"]."mismatchedWorkspaceDependencies": []}} ✨ Donein0.04 s.Copy the code
  • yarn workspaces run <command>Execute in all workspaces<command>

Equivalent commands in lerna: lerna run

  • yarn workspace <workspace> <command>In the specified<workspace>To run the yarn command

lerna

GitHub – LERna: A tool for managing JavaScript projects with multiple packages

  • Common global parameters

    • –since

  • Common Commands

    • lerna bootstrap

    • lerna add

      • –scope

    • lerna publish

      • from-package

    • lerna run

    • lerna exec

Monorepo Best practices

Use the typescript

Tsconfig. json in each workspace inherits the tsconfig.json in the root directory to promote the common configuration.

  • Install the typescript
yarn add -W -D typescript
Copy the code
  • Initialize the project
yarn tsc --init
Copy the code

Single test was performed using JEST

Official document: Getting Started · Jest

Each workspace uses its own JEST configuration

  • Install dependencies
yarn add -W -D jest @types/jest ts-jest
Copy the code
  • Initial Configuration
yarn jest --init
Copy the code
  • Support for TS, see Github-Kulshekhar/TS-Jest: TypeScript preprocessor with Sourcemap support for Jest
{
    "preset": "ts-jest",
}
Copy the code

Use ESLint & Prettier to unify the code style

More view documentation: www.robertcooper.me/using-eslin…

  • Install Lint dependencies
yarn add -W -D eslint @typescript-eslint/parser @typescript-eslint/eslint-plugin eslint-config-prettier eslint-config-standard eslint-config-standard-with-typescript eslint-plugin-import eslint-plugin-jest eslint-plugin-node  eslint-plugin-prettier eslint-plugin-promise eslint-plugin-standardCopy the code
  • Add the following configuration

.eslintrc.json

module.exports = {
  parser: '@typescript-eslint/parser',
  parserOptions: {
    ecmaVersion: 2020,
    sourceType: 'module',
    tsconfigRootDir: __dirname,
    project: [
      './packages/**/tsconfig.json'
    ]
  },
  env: {
    node: true
  },
  plugins: [
    '@typescript-eslint',
    'jest',
    'prettier'
  ],
  extends: [
    'eslint:recommended',
    'standard-with-typescript',
    'prettier/@typescript-eslint',
    'plugin:jest/recommended',
    'plugin:prettier/recommended'
  ],
  rules: {
    '@typescript-eslint/explicit-function-return-type': 'off',
    '@typescript-eslint/triple-slash-reference': 'off',
    '@typescript-eslint/strict-boolean-expressions': 'off'
  }
}
Copy the code
  • Vscode support,.vscode/setting.json
{
  "editor.codeActionsOnSave": {
    "source.fixAll.eslint": true
  }
}
Copy the code
  • Prettier configuration

.prettierrc.json

{
  "semi": false,
  "tabWidth": 2,
  "singleQuote": true,
  "trailingComma": "none"
}
Copy the code

Can be modified according to their own taste

  • Git Hook configuration & Lint-staged
yarn add -W -D husky lint-staged
Copy the code
  • The configuration is as follows:

.lintstagedrc

{
  "packages/**/*.{js,ts,jsx,tsx}": [
    "eslint --fix"
  ]
}
Copy the code

.huskrc.json

{"hooks": {// commitlint set git hook "commit- MSG ": Commitlint-e HUSKY_GIT_PARAMS", // note here --since HEAD https://github.com/lerna/lerna/tree/master/core/filter-options#--since-ref "pre-commit": "lerna exec --concurrency 1 --stream lint-staged --since HEAD" } }Copy the code

Automate semantic version management with CommitLint

Official documentation: commitlint-Lint commit messages

  • Install dependencies
yarn add -W -D @commitlint/cli @commitlint/config-conventional @commitlint/config-lerna-scopes
Copy the code
  • Adding a Configuration File

.commitlintrc.json

{
  "extends": [
    "@commitlint/config-conventional",
    "@commitlint/config-lerna-scopes"
  ]
}
Copy the code

Use PLOP for boilerplate code configuration to speed up development efficiency

Official document: GitHub – plopjs/plop: Consistency Made Simple

Specific reference code library implementation: Github-dancon /monorepo

  • Install dependencies
yarn add -W -D plop
Copy the code
  • The configuration fileplopfile.js
module.exports = function(plop) {
    // ...
}
Copy the code

  • Commonly used method

    • setGenerator

    • setHelper

  • The project root directory is configured with the following script

{
    "scripts": {
        "gen": "plop"
    }
}
Copy the code

Then run yarn gen [template-name] to create the template you configured, or if you do not specify [template-name], all templates you configured will be listed. For more information, please refer to the official documentation.

References

  • Why Lerna and Yarn Workspaces is a Perfect Match for Building Mono-repos — a Close Look at Features and Performance — Sebastian Weber — Frontend Developer and UX Guy

  • Monorepo in the Wild

  • GitHub – lerna/lerna: A tool for managing JavaScript projects with multiple packages.

  • yarn workspace | Yarn

  • GitHub – plopjs/plop: Consistency Made Simple

  • GitHub – SBoudrias/Inquirer.js: A collection of common interactive command line user interfaces.