Front-end modularization has become the mainstream today, inseparable from the contribution of a variety of packaging tools. Webpack, Rollup, and up-and-coming Parcel are all mentioned in the community, and there are plenty of configuration analyses for each of them. To avoid becoming a “configuration engineer,” we need to understand how the packaging tool works. Once we understand the core principles, we will be more comfortable using the tool.
This article is based on parcel core developer @Ronami’s open source project MiniPack, and has added more understanding and explanation to its very detailed notes to make it easier for readers to understand.
1. Core principles of packaging tools
As the name implies, the packaging tool is responsible for a number of scattered small modules, according to certain rules into a large module tool. At the same time, the packaging tool takes care of the dependencies between modules so that the large module can be run on the appropriate platform.
The packaging tool starts with an entry file, analyzes the dependencies in it, and then further analyzes the dependencies in the dependencies, repeating the process until the dependencies are untangled.
As you can see from the above description, the core part of the packaging tool is to deal with the dependencies between modules, and minipack and this article are also focused on module dependencies.
For simplicity, the Minipack project uses ES Modules specification directly. Next we create three new files and create dependencies between them:
/* name.js */
export const name = 'World'Copy the code/* message.js */ import { name } from './name.js' export default `hello ${name}! `Copy the code/* entry.js */
import message from './message.js'
console.log(message)Copy the codeTheir dependencies are simple: Entry.js → message.js → name.js, where entry.js will be the entry file for the packaging tool.
However, this dependency is only as we understand it, and if machines are to understand this dependency, they need some means.
2. Dependency analysis
Create a new js file called minipack.js and first introduce the necessary tools.
/* minipack.js */
const fs = require('fs')
const path = require('path')
const babylon = require('babylon')
const traverse = require('babel-traverse').default
const { transformFromAst } = require('babel-core')Copy the codeNext, we’ll write a function that takes a file as a module, reads its contents, and analyzes all of its dependencies. Of course, we could match the import keyword in the module file with a regular, but this would be very inelegant, so we could use the JS parser Babylon to convert the file contents into an abstract syntax tree (AST) and retrieve the desired information directly from the AST.
Once you have an AST, you can traverse the AST with babel-traverse to pick up the key “dependency declarations” and store them in an array.
Finally, babel-core’s transformFromAst method and babel-Preset -env are used to transform ES6 syntax into ES5 syntax that the browser can recognize, and an ID is assigned to the JS module.
Function createAsset (filename) {const content = fs.readfilesync (filename, 'utF-8 ') // convert to AST const AST = Babylon. Parse (content, {sourceType: 'module',}); Const dependencies = [] // traverse(ast, {ImportDeclaration: ({ node }) => { dependencies.push(node.source.value); }}) // Convert ES6 syntax to ES5 const {code} = transformFromAst(ast, null, {presets: ['env'],}) const ID = ID++Copy the codeRun createAsset(‘./example/entry.js’) and the output looks like this:
{ id: 0, filename: './example/entry.js', dependencies: [ './message.js' ], code: '"use strict"; \n\nvar _message = require("./message.js"); \n\nvar _message2 = _interopRequireDefault(_message); \n\nfunction _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }\n\nconsole.log(_message2.default); '}Copy the codeThe entry. Js file has become a typical module, and dependencies have been analyzed. The next step is to recurse and analyze the “dependencies of dependencies”, which is the dependency atlas discussed in the next section.
3. Establish dependency atlas
Create a new function called createGragh(), pass in the path of an entry file, and parse this file to define a module with createAsset().
Next, to be able to do dependency analysis on modules one by one, we maintain an array, first passing in the first module and analyzing it. When this module has other dependencies, it adds them to the array and continues to analyze the new modules until all dependencies and “dependencies within dependencies” have been fully analyzed.
At the same time, it is necessary to create A mapping attribute for the module to store the dependencies among modules, dependencies and dependency ids. For example, “module A with ID 0 depends on module B with ID 2 and module C with ID 3” can be expressed as follows:
{
0: [function A () {}, { 'B.js': 2, 'C.js': 3 }]
}Copy the codeOnce you know what’s going on, you can start writing functions.
Function createGragh (Entry) {const mainAsset = createAsset(Entry) // Maintain an array, Pass in the first module const queue = [mainAsset] // Walk through the list to see if each module has any other dependencies, For (const asset of queue) {asset-. mapping = {} // Since the dependency path is relative to the current module, So want to handle the relative path to the absolute path const dirname = path. The dirname (asset. The filename) / / traverse the current module dependencies and continue to analyze asset. The dependencies. The forEach (relativePath Const absolutePath = path.join(dirname, absolutePath) RelativePath) const Child = createAsset(absolutePath) asset-. Mapping [relativePath] = Queue.push (child)})} return queue}Copy the codeThere may be readers who are interested in the for… of … The queue.push loop is a bit of a puzzle, but try this code to figure it out:
var numArr = ['1', '2', '3'] for (num of numArr) { console.log(num) if (num === '3') { arr.push('Done! ')}}Copy the codeTry running createGraph(‘./example/entry.js’) and you’ll see the following output:
[ { id: 0, filename: './example/entry.js', dependencies: [ './message.js' ], code: '"use strict";\n\nvar _message = require("./message.js");\n\nvar _message2 = _interopRequireDefault(_message); \n\nfunction _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }\n\nconsole.log(_message2.default);', mapping: { './message.js': 1 } }, { id: 1, filename: 'example/message.js', dependencies: [ './name.js' ], code: '"use strict";\n\nObject.defineProperty(exports, "__esModule", {\n value: true\n});\n\nvar _name = require("./name.js");\n\nexports.default = "Hello " + _name.name + "!";', mapping: { './name.js': 2 } }, { id: 2, filename: 'example/name.js', dependencies: [], code: '"use strict";\n\nObject.defineProperty(exports, "__esModule", {\n value: true\n});\nvar name = exports.name = \'world\';', mapping: {} } ]Copy the codeNow that the dependency atlas is built, it’s time to package them into a single, runnable file.
4. Pack
The dependency atlas generated in the previous step will be loaded through the CommomJS specification. Due to space, this article will not extend CommomJS specification, interested readers can refer to @Ruan Yifeng teacher’s article “Browser loading CommonJS module principle and implementation”, said very clearly. Function () {})(), module, exports, and require variables.
The next step is to use string concatenation to build code blocks according to the specification.
function bundle (graph) {
let modules = ''
graph.forEach(mod => {
modules += `${mod.id}: [
function (require, module, exports) { ${mod.code} },
${JSON.stringify(mod.mapping)},
],`
})
const result = `
(function(modules) {
function require(id) {
const [fn, mapping] = modules[id];
function localRequire(name) {
return require(mapping[name]);
}
const module = { exports : {} };
fn(localRequire, module, module.exports);
return module.exports;
}
require(0);
})({${modules}})
`
return result
}Copy the codeFinally, run the bundle(createGraph(‘./example/entry.js’)) and output the following:
(function (modules) { function require(id) { const [fn, mapping] = modules[id]; function localRequire(name) { return require(mapping[name]); } const module = { exports: {} }; fn(localRequire, module, module.exports); return module.exports; } require(0); ({}) 0: [ function (require, module, exports) { "use strict"; var _message = require("./message.js"); var _message2 = _interopRequireDefault(_message); function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; } console.log(_message2.default); }, { "./message.js": 1 }, ], 1: [ function (require, module, exports) { "use strict"; Object.defineProperty(exports, "__esModule", { value: true }); var _name = require("./name.js"); exports.default = "Hello " + _name.name + "!"; }, { "./name.js": 2 }, ], 2: [ function (require, module, exports) { "use strict"; Object.defineProperty(exports, "__esModule", { value: true }); var name = exports.name = 'world'; }, {}, ], })Copy the codeThis code will run directly in the browser, printing “Hello world!” .
At this point, the entire packaging tool is complete.
5. Summarize
After the above steps, we can know a module packaging tools, the first step can be started from the entrance to the file, to rely on the analysis, the second step is to rely on its all depend on recursive again analysis, the third step build module relies on atlas, the last step according to the dependence on atlas using CommonJS standard build the code. Once you understand the purpose of each step, you can understand how a packaging tool works.
Finally, thanks to @ronami for his open source project Minipack, the source code has a more detailed annotation, which is well worth reading.