What is a bundler

There are a number of bundlers on the market, most notably Webpack, as well as Browserify, Rollup, and Parcel. Although the Bundler has evolved a variety of features, they all share a common purpose: to bring a modular approach to the front end, better dependency management, and better engineering.


There are two types of modular systems that are most common today:

ES6 Modules:

// import module import _ from 'lodash'; // export module export default someObject;

CommonJS Modules:

// import module const _ = require('lodash'); // exports module.exports = someObject;

Dependency Graph

A common project needs an entry point. From this entry point, Bundler finds all the modules the project depends on and forms a dependency map. With this dependency map, Bundler then packages all the modules into a single file.

Dependency diagram:

Bundler implements the idea

To implement a Bundler, there are three main steps:

  1. Parse a file and extract its dependencies
  2. Recursively extract the dependencies and generate the dependency graph
  3. Package all dependent modules into a single file

This article uses a small example to show how to implement a Bundler. As shown in the following figure, there are three JS files: an entry file, an entry.js dependency file, greetings.js, and a greetings.js dependency file, named.

The contents of the three files are as follows:

Entry. Js:

import greeting from './greeting.js';


The greeting. Js:

import { name } from './name.js'; export default `hello ${name}! `;

Name. Js:

export const name = 'MudOnTire';

Realize bundler

We’ll start by creating a new Bundler.js file where the main Bundler logic is written.

1. Introduce JS Parser

In our implementation, we first need to be able to parse the contents of a JS file and extract its dependencies. It is possible to read the contents of the file as strings and use the regex to get import and export statements, but this is not elegant and efficient. A better way is to use JS Parser to parse the contents of the file. JS Parser parses JS code and converts it into a high-level model of an Abstract Syntax Tree (AST), which breaks down JS code into a tree structure from which more details of the code’s execution can be retrieved.

In the AST Explorer, you can see the result of parsing JS code into an abstract syntax tree. For example, the greeting.js content is parsed using the Acron Parser as follows:

You can see that the abstract syntax tree is actually a JSON object, with each node having a type attribute and the result of parsing an import, export statement, and so on. Turning the code into an abstract syntax tree makes it easier to extract key information.

Next, we need to introduce a JS Parser into the project. We chose Babylon (which is also Babel’s in-house JS Parser and currently exists on Babel’s main repository as @Babel/Parser).

Install the Babylon:

npm install --save-dev @babel/parser

Or yarn:

yarn add @babel/parser --dev

Introduce Babylon to Bundler.js:

Bundler. Js:

const parser = require('@babel/parser');

2. Generate abstract syntax trees

With the JS Parser, it is easy to generate the abstract syntax tree. All we need to do is get the contents of the JS source file and pass it to the Parser for parsing.

Bundler. Js:

const parser = require('@babel/parser'); const fs = require('fs'); /** * Get the filename */ function getAST(filename) {const content = fs.readFileSync(filename, 'utf-8'); const ast = parser.parse(content, { sourceType: 'module' }); console.log(ast); return ast; } getAST('./example/greeting.js');

Execute Node Bundler.js and the result is as follows:

3. Dependency resolution

After the abstract syntax tree is generated, we can find the dependencies in the code. We can write our own query method to find them recursively, or we can use @babel/traverse to find them. The @babel/traverse module maintains the state of the whole tree and is responsible for replacing, deleting and adding nodes.

Install @ Babel/traverse by:

npm install --save-dev @babel/traverse

Or yarn:

yarn add @babel/traverse --dev

Use @babel/traverse for easy access to import nodes.

Bundler. Js:

const traverse = require('@babel/traverse').default; /** * import Declaration */ Function getReports (AST) {traverse(AST, {importDeclaration: ({ node }) => { console.log(node); }}); } const ast = getAST('./example/entry.js'); getImports(ast);

Execute Node Bundler.js and the result is as follows:

From this we can get the dependent modules in Entent.js and the path to those modules. Modify the getReports method slightly to get all dependencies:

Bundler. Js:

function getImports(ast) { const imports = []; traverse(ast, { ImportDeclaration: ({ node }) => { imports.push(node.source.value); }}); console.log(imports); return imports; }

Execution Result:

Finally, we encapsulate the method to generate unique dependency information for each source file, including the ID of the dependent module, the relative path of the module, and the dependencies of the module:

let ID = 0;

function getAsset(filename) {
  const ast = getAST(filename);
  const dependencies = getImports(ast);
  const id = ID++;
  return {

const mainAsset = getAsset('./example/entry.js');

Execution Result:

4. Generate the Dependency Graph

Then, we need to write a method to generate the dependency graph that takes the entry file path as an argument and returns an array containing all the dependencies. Dependency diagrams can be generated either recursively or queuing. In this paper, we use queues to iterate over the asset objects in the queue. If the dependencies of the asset object are not empty, we will continue to generate and queue assets for each dependency, and add the mapping attribute to each asset to record the relationship between dependencies. This process continues until the elements in the queue are fully traversed. The specific implementation is as follows:


Const mainAsset = getAsset(entry); const mainAsset = getAsset(entry); const mainAsset = getAsset(entry); const queue = [mainAsset]; for (const asset of queue) { const dirname = path.dirname(asset.filename); asset.mapping = {}; asset.dependencies.forEach((relPath, index) => { const absPath = path.join(dirname, relPath); const child = getAsset(absPath); asset.mapping[relPath] = child.id; queue.push(child); }); } return queue; }

The generated dependencies are as follows:

5. Packaging

Finally, we need to package all the files into one file based on the dependency diagram. There are a few key points to this step:

  1. The packaged file needs to be able to run in a browser, so the ES6 syntax in the code needs to be compiled by Babel first
  2. In the browser runtime environment, the compiled code still needs to implement the reference between modules
  3. After merging into a single file, the scopes of the different modules still need to remain separate

(1). Compile the source code

First install Babel and introduce:

npm install --save-dev @babel/core

Or yarn:

yarn add @babel/core --dev

Bundler. Js:

const babel = require('@babel/core');

And then modify the getAsset methods, here we use the Babel. TransformFromAstSync () method for the compilation of the abstract syntax tree, compiled into the browser can execute JS:

function getAsset(filename) { const ast = getAST(filename); const dependencies = getImports(ast); const id = ID++; / / compile const. {code} = Babel transformFromAstSync (ast, null, {presets: [' @ Babel/env]}); return { id, filename, dependencies, code } }

The content of the generated dependency diagram is as follows:

You can see the require(‘./greeting.js’) syntax in the compiled code, and the browser does not support the require() method. So we also need to implement the require() method to implement the reference between modules.

(2). Module reference

First, the packaged code needs its own independent scope so as not to contaminate other JS files, so use the IIFE wrapper here. We can start by outlining the structure of the bundling method by adding a bundle() method to Bundler.js:

Bundler. Js:

/** * @param {Array} graph dependency */ function bundle(graph) {let modules = "; Iife Graph. Foreach (mod => {modules += '${mod.id}') :[function (require, module, exports) { ${mod.code}}, ${JSON.stringify(mod.mapping)} ],` }) // return ` (function(){})({${modules}}) `; }

Let’s first look at the result after executing the bundle() method (beautified with js-beautify and cli-highlight for easy reading) :

Now that we need to implement references between modules, we need to implement the require() method. The realization idea is as follows: When require(‘./ greetings.js ‘) is called, go to the mapping to find the module ID corresponding to./ greetings.js, find the corresponding module through the ID, call the module code to return exports, and finally package and generate main.js file. The full implementation of the bundle() method is as follows:

Bundler. Js:

/** * @param {Array} graph dependency */ function bundle(graph) {let modules = "; Iife Graph. Foreach (mod => {modules += '${mod.id}') :[function (require, module, exports) { ${mod.code}}, ${JSON.stringify(mod.mapping)} ],` }) const bundledCode = ` (function (modules) { function require(id) { const [fn, mapping] = modules[id]; function localRequire(relPath) { return require(mapping[relPath]); } const localModule = { exports : {} }; fn(localRequire, localModule, localModule.exports); return localModule.exports; } require(0); })({${modules}}) `; fs.writeFileSync('./main.js', bundledCode); }

Finally, let’s run the contents of main.js in our browser and see what we get:

A simple version of Webpack is done!

This article source: https://github.com/MudOnTire/…