In our daily development, Modules are basically used, whether it is CommonJS of Node.js, ES Modules of ES6, or even older AMD, UMD, etc. The introduction of Modules brings a lot of convenience to our development and solves a lot of problems for us. But what exactly is it?

Let’s start from the history of JavaScript Modules, to the latest ES Modules, a new understanding of Modules.

1. History of JavaScript modules

1.1 Vanilla JS (1995~2009)

When JavaScript was developed, there was no standard for modules, because JavaScript was designed to be a toy Script for simple interactions in the browser. However, with the rapid development of the Internet, people are no longer satisfied with simple interaction, and the complexity of code is also increasing, and the maintenance difficulty is becoming higher and higher.

So what does maintenance mean? This refers to maintenance variables. Because as projects iterate, collaborative development is inevitable. In the early days of JS, all variables are written in global scope, so what is the likely problem? Overwriting, tampering, and deleting variables is a headache. It’s quite possible that one day your functionality is going wrong because one of your variables was deleted by another developer.

Therefore, the original purpose of the introduction of modules is to understand the control of variables. There are other benefits, such as encapsulation of code, reuse, and so on.So how did developers achieve the same effect as modules in the early days without the support of module standards? There are two ways.

1.1.1 Object Literal Pattern

Use JS built-in objects to control variables:

function Person(name) {
  this.name = name;
}

Person.prototype.talk = function () {
  console.log("my name is".this.name);
};

const p = new Person("anson");
p.talk();
Copy the code

This allows you to keep the variables inside the object with a new Person.

1.1.2 IIFE (Immediately Invoked Function Expression)

We know that in JavaScript we have the concept of Scope. Variables in a Scope are only visible within the Scope. Before ES6, there were only two types of scopes:

Global Scope
Function Scope

As mentioned above, the control of variables must be as small as possible, so there is no doubt that writing variables in functions is the best way. However, this raises the question of how variables in a function are made available for external use.There is no good solution to this problem in the beginning, you have to expose variables to the global scope, like classic jQuery.Developers often use IIFE to implement:

// lib.js
(function() {
  const base = 10;
  this.sumDOM = function(id) {
    / / rely on jQuery
    return base + +$(id).text();
  }
})();
Copy the code

Introduce lib.js into HTML:

// index.html
<html>
  <head>
    <script src="/path/to/jquery.js"></script>
    <script src="/path/to/lib.js"></script>
  </head>
  <body>
    <script>
      window.sumDOM(20);
    </script>
  </body>
</html>
Copy the code

But IIFE has several problems:

At least one variable contaminates the global scope;
Dependencies between modules are vague and unclear (lib.jsDependencies are not intuitively visiblejquery.js);
Load order is not guaranteed and is not easy to maintain (must be guaranteedjquery.jsMust be inlib.jsPreloading completed, otherwise an error will be reported.

So there is a strong need for a module standard for JavaScript to address these issues.

1.2 Non-native Module Format & Module Loader (2009~2015)

Because modules can solve these problems, developers try to design some non-native Module standards such as CommonJS, Asynchronous Module Definition (AMD), Universal Module Definition (UMD), This can then be achieved by using corresponding Module Loaders such as CJS-Loader, RequireJS, and SystemJS. Let’s take a look at a few popular non-native Module standards.

1.2.1 CommonJS (CJS)

In 2009, Kevin, an engineer from Mozilla, proposed CommonJS, a modular standard for JavaScript running outside the browser, mainly on the server side such as Node.js. This worked well and was subsequently used in browser module development, but since browsers did not support CommonJS, transpilers such as Babel were required to convert the code to ES5 to run on the browser.

CommonJS features require to import dependencies and exports to export interfaces.

// lib.js
module.exports.add = function add() {};

// main.js
const { add } = require("./lib.js");
add();
Copy the code

1.2.2 AMD

Because CommonJS is designed to be applied on the server side, modules are loaded and executed synchronously (because local files have fast IO). But synchronization is not friendly to browsers, where module files are loaded over the network, and single-thread blocking on module loads is unacceptable. So in 2011 someone came up with AMD, which is compatible with CommonJS and supports asynchronous loading.AMD is characterized by the use ofdefine(deps, callback)To load modules asynchronously.

// Calling define with a dependency array and a factory function
define(['dep1'.'dep2'].function (dep1, dep2) {
    //Define the module value by returning a value.
    return function () {};
});
Copy the code

1.2.3 UMD

Due to the popularity of CommonJS and AMD, UMD module standard was proposed. UMD is compatible with AMD, CommonJS and Global Variable formats by detecting different environment features.

// UMD
(function (root, factory) {
  if (typeof define === 'function' && define.amd) {
    // AMD
    define(['jquery'.'underscore'], factory);
  } else if (typeof exports= = ='object') {
    // Node, CommonJS-like
    module.exports = factory(require('jquery'), require('underscore'));
  } else {
    // Browser globals (root is window)
    root.returnExports = factory(root.jQuery, root._);
  }
}(this.function ($, _) {
  // methods
  function a(){};    // private because it's not returned (see below)
  function b(){};    // public because it's returned
  function c(){};    // public because it's returned
  // exposed public methods
  return {
    b: b,
    c: c
  }
}));
Copy the code

Because of the compatibility of UMD, many libraries provide versions of UMD.

1.3 ESM (2015~ Now)

With the gradual normalization and standardization of ECMAScript, ES6 (ES 2015) was finally released in 2015. In this version update, THE JS module standard, namely ES Modules, was formulated. ES Modules declare dependencies with import. Export Specifies an interface.

// lib.mjs
const lib = function() {};
export default lib;

// main.js
import lib from './lib.mjs';
Copy the code

As of 2018, most major browsers already support ES Modules, which declare AN ESM type in HTML by adding a type=”module” attribute to

There are a few caveats to using ES Modules in HTML:

The strict mode is enabled by default"use strict";
The defaultdeferLoad execution;
Enabled by defaultCORSCross domain;
In the same document, the same module is only loaded and executed once;

With the release of the ES Modules standard, the surrounding ecosystem of JS is slowly getting closer to ES Modules. Node.js added support for ES Modules in 14.x; Module Bundler Such as Rollup uses ES Modules as the default Module standard. Deno, TypeScript, etc.

The main module standards that are still in use today are CJS and ESM. CJS exists mainly because of Node’s history. The working principle of ESM is introduced and compared with CJS.

2. Working principle of ESM

Before introducing how ES Modules works, let’s understand a few concepts.

Module Scope

As we all know, ES6 introduces a new scope:Block ScopeBut there is one moreModule Scope, which manages variables within the module. Unlike function scope, in module scope you need to explicitly specify the exported variable, which is also called aexport; You also need to explicitly specify the variable to import, which is also called aimport. So you no longer need to pollute the global scope.Because the dependencies between modules are explicit, you don’t have to worry if your module is not preloaded because jquery will tell you at compile time.

Module Record

When we use modules, we are actually building a module dependency graph. You pass a module file as an Entry Point, and the JS engine recursively queries, downloads, and parses submodules based on the module’s import declaration.

Here main.js is the entry and then relies on the other two submodules, counter.js and display.js.Parsing refers to parsing a module file into a data structureModule Record, Module Record records the contents of the Moduleimport,export,codeFor subsequent Linking and Evaluation.

Module Environment Record

When the JS engine executes into a scope, it creates an Environment Record bound to that scope to store variables and functions within that scope. In addition to the top-level variable declarations in the Module, the Module Environment Record also stores the import bound variables in the Module.

Environment Record has a very important field[[OuterEnv]]Used to point to the external Environment Record, which is very similar to the prototype chain, ending innull.This is shown in the figure abovelib-a.js 与 lib-b.jsAre two independent modules, moduleEnvironmentRecord-Lib-A and ModuleEnvironmentRecord-lib-b, respectively[[OuterEnv]]Both point to GlobalEnvironmentRecord, which separates variables between modules.

The working process of ES Modules can be divided into three stages:

Construction – Query, download and parse modules as Module Record;
Linking– Create an Environment Record and associate it between modulesimport,exportRelationship;
Evaluation – Execute the top-level code and populate the Environment Record.

It is commonly said that ESM is executed asynchronously, because the three stages are independent and separable. However, this does not necessarily mean that it needs to be implemented asynchronously. It can also be executed synchronously, for example, in CJS.

Because in theESM specIt only explains how to parse a Module Record; How to do Linking between modules; How to perform Evaluation of modules. However, there is no mention of how to get the module files, which are loaded by different loaders in different runtime environments. For browsers, inHTML specIs used in the asynchronous loading mode.Loader is not only responsible for loading modules, it is also responsible for calling ESM methods such asParseModule,Module.Link,Module.Evaluate. Loader controls the order in which these methods are executed.

2.1 Construction

The Construction phase is divided into three steps:

Find the module path, also called module resolution;
Get the module file (download from the network or load from the file system);
Parse the Module file as Module Record;

Loader is responsible for addressing and downloading modules. First we need an entry file, which is usually one in HTML<script type="module">To represent a module file (usually used in Node)*.mjsTo represent a module file or modify package.json"type": "module")How does a module find its next submodule? That’s what you need to get throughimportDeclare the statement inimportA part of the declaration statement is calledmodule specifierThis tells loader how to find the address of the next submodule.Pay attention tomodule specifierThere are different methods of interpretation in different environments (browser, Node), and the process of interpretation is also calledmodule resolution. For example, only urls are supported in the browsermodule specifier; Node supports more than thatbare module specifierThat’s what we always writeimport moment from "moment";. The W3C is also moving forwardimport mapsFeatures to supportbare module specifier.

You can only know which submodules the current Module depends on when you have resolved the current Module as a Module Record. Then you need to resolve, fetch, parse, and resolve submodules. Parsing -> fetching -> parsing of the process continues, as shown in the figure below:If the main thread is waiting for each module file to be downloaded throughout the process, the entire task queue will hang. This is because your download in the browser is slow, which is why the ESM Spec splits module loading into three phases.

Phase splitting is also a major difference between CJS and ESM, because CJS loads local files and naturally does not need to consider IO issues. This means that Node blocks the main thread to load the module and then executes Linking and Evaluation synchronously.The above code executes torequireThen you need to load the submodule, immediately switch to the FETCH submodule, and continue to execute the evaluate submodule, all in sync. That’s why in Node, you can usemodule specifierUse variables in.

This is not the case for the ESM, because the entire module dependency diagram, including resolving, fetching, and parsing of all modules, needs to be built before the ESM can execute the Evaluation. Therefore, the ESM cannot use variables in Module Specifier.

One of the benefits of this is that Module Bundler like Rollup and Webpack can do static analysis of the ESM at compile time and Tree Shaking can remove dead code.If you really want to use variables in the ESMmodule specifier, then you can usedynamic import import(${path}/foo.js)To import a new module, the new module entry automatically creates a new module dependency graph.Although it is a new Module dependency graph, it does not create a new Module Record for loader to useModule MapTrack and cache the global Module Record. This ensures that the module file is only fetched once. There is a separate Module Map for each global scope, so each iframe has its own Module Map.

Think of a Module Map as a simple key/value mapping object. For example, the first loaded module will be marked asfetching, and then initiate the request and proceed to fetch the next module file.To understand the relationship between Document and Module Map, look at the following figure:Document and Module Map have a one-to-one relationship. Main.js has its own Module Map. Iframe -a and iframe-b have their own Module Map. So even though their internal dependent module addresses are the same, they still repeatedly request downloads.Ok, so after downloading the file, THE JS engine will parse the Module file into Module Record, saving the import, export, code and other information in the Module.The Module Record is cached in the Module Map.

2.2 Linking

After all Module records have been parsed, the JS engine then needs to link all modules. JS engine takes Module Record of entry file main.js as the starting point, recursively links modules in depth-first order, and creates a Module Environment Record for each Module Record. Use to manage variables in the Module Record.

How do you link? JS engine will create Module Environment Record for all submodules counter. JS and display. JS under the current Module main.jsexportVariable is bound to allocate memory space for it.Then control goes back to the previous level, which is the current module main.js, on main.jsimportNotice here in main.jsimportPoint to the memory location in count.js and display.jsexportThe variables point to the same memory location, thus linking the relationship between the parent and child modules.But CJS is different at this point. In CJS, the wholemodule.exportsObject is copied.This means that the Exporting Module changes the variable values later, and the importing Module does not automatically update.

Instead, THE ESM uses a technique called Live Bindings. The parent and child modules point to the same memory location, so the Exporting Module modifies the variable values, and the importing Module is immediately updated.

It’s important to note that onlyexportingModule is requiredexportTo change the value of the variable,importingYou can’t change a module. To say theexportingModules have read and write permissions, andimportingThe module has read permission only.One reason to use Live Bindings is that it helps to tie all modules together without having to run any code. This is helpful when we are evaluating against Cyclic dependencies.

Now it’s time to execute the code and populate the memory above.

2.3 Evaluation

After the modules are linked to each other, the JS engine implements this by executing top-level code in the modules, so yourimport,exportStatements cannot be written inside functions.But executing top-level code can have side effects, such as sending network requests, so you don’t want to execute the same module more than once. This is why Module Map is used to cache Module records globally. If the status of a Module Record isevaluated, the next execution will be automatically skipped, ensuring that a module is executed only once. As with the Linking phase, depth-first traversal is performed on the Module Record.

The problem of dependency loop mentioned at the Linking end is usually a complicated dependency loop. Here is a simple example to illustrate:Main.js and counter.js loop around each other. Let’s first look at the dependency loop problem in CommonJS:First, main.js executes torequire("./counter.js")Then go to counter.js and execute to get main.jsmessage, and this isundefinedSo counter.js is copiedundefined.After executing counter. Js (note that we set a setTimeout at the end to see if the message is automatically updated), we return control to main.js and continue executing the codemessageThe assignment for"Eval complete".But because it’s in CommonJSimportThe value of the variable is yesexportVariable value, so counter. JsmessageIt doesn’t update.ES Modules uses live Bindings, so it will be updated automatically in counter.jsmessageThe value of the.

3. Mixed use between CJS and ESM

For historical reasons, most packages on NPM were written in CJS, but with the advent of ESM, developers began to use ESM to write modules. In order to maximize reuse of packages on NPM, it is inevitable to import CJS in ESM. CJS cannot be imported to THE ESM due to the difference in module loading modes, but THE ESM can import CJS.

Although THE ESM can import CJS, its use is somewhat limited.

3.1 The ESM supports only default import CJS

The ESM supports CJS default import, but does not support named import.

import pkg from 'lib.cjs'; // work
import { fn1, fn2 } from 'lib.cjs'; // error
Copy the code

Why is that? Combined with the above working principle of ESM, ESM carries out static analysis on module variables, while CJS module variables are dynamically calculated. So how can the ESM compute the CJS module variables while the first phase of Construction has not yet executed the code?

But in Node 14.13.0, NodeAdded support for CJS named exportCan support most CJS modules.Why most of them? The Node website was createdinstructions:

The detection of named exports is based on common syntax patterns but does not always correctly detect named exports. In these cases, using the default import form described above can be a better option.

Note that the detect keyword, which is based on the CJS module syntax for the text analysis of named exports, is not guaranteed to be correct. In this case, using default import is a better choice.

}}}}}}}}}}}}}}}}}}}}}}}}}}}} Or the module exports = the require (‘… ‘), here’s an example that can be analyzed:

// correct.cjs
exports.a = 1;
exports.b = 2;
if (Math.random() > 0.5) {
  exports.c = 3;
}

// main.mjs
import { a, b, c } from './correct.cjs';
// Execute main.mjs without exception
Copy the code

Examples that cannot be analyzed:

// wrong.cjs
// exports with TMP
const tmp = exports;
tmp.a = 1;
tmp.b = 2;
if (Math.random() > 0.5) {
  tmp.c = 3;
}

// main.mjs
import { a, b, c } from './wrong.cjs';
// An error occurs when executing main.mjs
Copy the code

Executing the above example will result in the following error:

file:///E:/javascript-modules/esm-app/dual/index.mjs:1
import { a, b, c } from "./lib.cjs";
         ^
SyntaxError: Named export 'a' not found. The requested module './lib.cjs' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:

import pkg from './lib.cjs';
const { a, b, c } = pkg;
Copy the code

You might think, who would write that? Unfortunately, a lot of famous libraries do. For examplelodash,chalk.For non-analysisnamed exportsModule, Node will advise us to use in the errordefault import, and then do the deconstruction, which is one more line of code:

CommonJS modules can always be imported via the default export, for example using:

import pkg from ‘./lib.cjs’; const { a, b, c } = pkg;

3.2 the use ofESM WrapperImplement named exports for CJS

If we really want to use named exports CJS in the ESM, we can provide an ESM Wrapper for the CJS. This Wrapper encapsulates a layer of code according to the Node error. Then re-export the specified variable once:

// lib.cjs
const tmp = exports;
tmp.a = 1;
tmp.b = 2;
if (Math.random() > 0.5) {
  tmp.c = 3;
}

// lib-esm-wrapper.mjs
import lib from "./lib.cjs";
export const { a, b, c } = lib;

// main.mjs
import { a, b, c } from "./lib-esm-wrapper.mjs";
console.log(a);
console.log(b);
console.log(c);
Copy the code

So when a user needs an ESM module and currently only has CJS modules, consider writing a simple ESM Wrapper to wrap it.

4. Write libraries that support multiple module formats

Sometimes when we write libraries, we want our libraries to support both CJS and ESM formats. You may be familiar with the module field of package.json, which is a conventional field. Module Bundler such as Webpack and Rollup checks whether a package supports ESM. Node does not recognize this field.

In Node 12+ we can configure packages to support different module files using the exports field of package.json. Node will return the module files depending on whether you load them using import or require:

// package.json
{
  "exports": {
   	"import": "./lib.mjs"."require": "./lib.cjs"}}// app.mjs
import { value } from "lib";
console.log("value from mjs", value);

// app.cjs
const value = require("lib").value;
console.log("value from cjs", value);
Copy the code

Refer to the article

ES modules: A cartoon deep-dive – Lin Clark
Node Modules at War: Why CommonJS and ES Modules Can’t Get Along – Dan Fabulich
Basics of Modular JavaScript – Christine Rohacz
Understanding ES6 Modules via Their History
tc39.es/ecma262
Html.spec.whatwg.org/multipage/w…
Node.js now supports named imports from CommonJS modules, but what does that mean?
How CommonJS is making your bundles larger

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

JavaScript Module Basics

1. History of JavaScript modules

1.1 Vanilla JS (1995~2009)

1.1.1 Object Literal Pattern

1.1.2 IIFE (Immediately Invoked Function Expression)

1.2 Non-native Module Format & Module Loader (2009~2015)

1.2.1 CommonJS (CJS)

1.2.2 AMD

1.2.3 UMD

1.3 ESM (2015~ Now)

2. Working principle of ESM

Module Scope

Module Record

Module Environment Record

2.1 Construction

2.2 Linking

2.3 Evaluation

3. Mixed use between CJS and ESM

3.1 The ESM supports only default import CJS

3.2 the use ofESM WrapperImplement named exports for CJS

4. Write libraries that support multiple module formats

Refer to the article

JavaScript Module Basics

1. History of JavaScript modules

1.1 Vanilla JS (1995~2009)

1.1.1 Object Literal Pattern

1.1.2 IIFE (Immediately Invoked Function Expression)

1.2 Non-native Module Format & Module Loader (2009~2015)

1.2.1 CommonJS (CJS)

1.2.2 AMD

1.2.3 UMD

1.3 ESM (2015~ Now)

2. Working principle of ESM

Module Scope

Module Record

Module Environment Record

2.1 Construction

2.2 Linking

2.3 Evaluation

3. Mixed use between CJS and ESM

3.1 The ESM supports only default import CJS

3.2 the use ofESM WrapperImplement named exports for CJS

4. Write libraries that support multiple module formats

Refer to the article

Related Posts

Still using Redux, want to try GraphQL and Apollo?

White’s server deployment journey

vue-cli+vuex+vueRouter+mock