Roid

Roid is a very simple packaging software, developed using Node.js. After reading this article, you can implement a very simple, but practical front-end code packaging tool.


If you don’t want to see the tutorial, go directly to the code (all comments) : click the address



Why roid?


We are faced with these front-end compilation tools every day, but I learned from a lot of conversations that not many people know the working principle behind these packaging software, hence the emergence of this project. Of course, you don’t need to know much about compilation principles or anything like that, but if you’re familiar with Node.js before this, you’ll have a pretty good understanding of front-end packaging.


Understanding the principle behind the packaging tool helps us to implement all kinds of amazing automation and engineering things, such as bidirectional binding of forms, self-created JavaScript syntax, the famous import plug-in of Ant Financial, and even automatic scanning and loading of front-end files, which can greatly improve our work efficiency.


Cut the crap. Let’s get right to it.


Start with an increment ID


const { readFileSync, writeFileSync } = require('fs') const path = require('path') const traverse = require('babel-traverse').default const { TransformFromAst, transform} = require('babel-core') let ID = 0 const currentPath = process.cwd()Copy the code



Id: global autoincrement ID, record the ID of each module loaded, we will all modules are marked with a unique identifier, so the autoincrement ID is the most effective and intuitive, how many modules, a statistic out.


Parse individual file modules


function parseDependecies(filename) { const rawCode = readFileSync(filename, 'utf-8') const ast = transform(rawCode).ast const dependencies = [] traverse(ast, { ImportDeclaration(path) { const sourcePath = path.node.source.value dependencies.push(sourcePath) } }) // Const es5Code = transformFromAst(AST, null, {presets: const es5Code = transformFromAst(AST, null, {presets: ['env']}).code // Remember our webpack-loader system? // The loader can be implemented by passing the file name and code into the loader for judgment, and even converting the user-defined behavior. // In webpack, we pass converted code between each loader and // instead of AST, We would have to convert the code -> AST on each loader. The time-consuming // parcel option is to send the AST directly instead of converting the code. Const customCode = loader(filename, es5Code) return {id: id ++, code: customCode, dependencies, filename } }Copy the code

First, we process each file. Because this is only a simple version of Bundler, we don’t worry about how to parse CSS, MD, TXT and so on. We concentrate on packaging JS files, because other files are different, and it’s easy to distinguish them by using file suffix. For this release, we’ll stick to JS.


The const rawCode = readFileSync(filename, ‘utF-8 ‘) function injects a filename, reads the text content of the file, and parses it through the AST. We used Babel’s Transform method to transform our original code into an abstract syntax tree (AST). You can check out https://astexplorer.net/, a visual site to see what the AST generates.


When we are done parsing dependencies, we can extract dependencies from the current file, which translates to “import XXXX from XXXX”. We place these dependencies in the dependencies array. Then export them uniformly.


Then traverse our code with traverse. Traverse function is a way to traverse an AST, provided by Babel-traverse, whose traverse mode is the classic visitor mode, which defines a series of visitors, When the TYPE === visitor name of the AST is encountered, the function of that visitor is entered. The AST node of type ImportDeclaration is our import XXX from XXXX, and we push the address to dependencies.


Finally, when exporting, remember that for each exported file module, we add + 1 to the global increment ID to ensure that each file module is unique.


Parse all files to generate dependency diagrams


Function parseGraph(entry) {// From entry, Const entryAsset = parseDependecies(path.resolve(currentPath, Entry)) // graph is an array, Const graph = [entryAsset] for (const asset of graph) {if (! Asset-idmapping) asset-idmapping = {} // Get the folder corresponding to the files in asset const dir = path.dirname(asset-.filename) // Each file will be parsed Create a dependencise, which is an array as described in the previous function // so we iterate over the array, DependencyPath = denpendencyAsset.id dependencyPath = denpendencyAsset.id Asset. Dependencies. ForEach (dependencyPath = > {/ / file module for absolute path, Such as import from ABC '. / world / / into/User/XXXX/desktop/xproject/world in the form of a const absolutePath = path. Resolve (dir, DependencyPath) const denpendencyAsset = parseDependecies(absolutePath) const id = Denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyasset. id = denpendencyAsset.id = denpendencyAsset.id = denpendencyAsset.id = denpendencyAsset.id = denpendencyAsset.id Find the code for this module, DependencyPath = DenPendencyAsset-.idMapping [dependencyPath] = DenPendencyAsset-.id graph.push(denpendencyAsset)} // Return graph}Copy the code


Next, we take a more advanced approach to the module. We’ve already written a parseDependecies function, so now we’re going to write a parseGraph function. We’re going to call the set of file modules a graph to describe all the dependencies of our project. The parseGraph starts from the entry and runs through all the files.


We’re using a for of loop here instead of a forEach loop, and the reason is because we’re in a loop and we’re pushing things into the graph, and the graph keeps getting bigger and bigger, and using for of keeps going through the loop until the graph doesn’t get pushed any more, and that means, All the dependencies have been resolved, and the graph array doesn’t grow any more, but forEach doesn’t work, it only iterates once.


In the for of loop, asset represents a module that has been parsed and it has things like filename, code, dependencies and so on. Asset-idmapping is an obscure concept, and we import every file. The import operation will then be converted to require. The path of require in every file will actually correspond to a numeric increment ID, and this increment ID is actually the ID that we set up at the beginning, by matching path-ID with key-value pairs, The reason why require is so verbose is that references between modules are often error-complex, which is exactly why the concept is so hard to explain.


Finally, the bundle is generated


Function build(graph) {// Our modules is a string let modules = "graph.forEach(asset => {modules += '${asset-. id}:[ function(require,module,exports){${asset.code}}, ${JSON.stringify(asset.idMapping)}, ],` }) const wrap = ` (function(modules) { function require(id) { const [fn, idMapping] = modules[id]; function childRequire(filename) { return require(idMapping[filename]); } const newModule = {exports: {}}; fn(childRequire, newModule, newModule.exports); return newModule.exports } require(0); })({${modules}}); } return wrap} // This is the simplest implementation of a loader function loader(filename, Code) {if (/index/.test(filename)) {console.log('this is loader ')} return code} // Finally we export our bundler module.exports  = entry => { const graph = parseGraph(entry) const bundle = build(graph) return bundle }Copy the code



Now that we’re done collecting graph, it’s time to wrap up our actual code, which uses a lot of string manipulation, and don’t be surprised why code and strings can be mixed up, and if you look at our code outside of writing code, code is actually strings, But it is organized by special language form, for script language JS, string splicing into code, and then run, this kind of operation is very common in the front end, I think, this kind of thinking transformation, is the first step to have automation, engineering.


We will be out all the asset in the graph, and then use the node. Js manufacturing module will approach to a code wrapped up, before I had a “skilled and magical craftsmanship: teach you how to implement” https://zhuanlan.zhihu.com/p/34974579


Function (require,module,exports){} function(require,module,exports){} function(require,module,exports){} function(require,module,exports){} function(require,module,exports){} This is why we can use these three things everywhere, because our code for each file will be wrapped in a function like this, but what’s weird about this code is that we’ve wrapped the code in a 1:[…] , 2: […]. When we finally import the module, we will add a {} to the string, which becomes {1:[…]. , 2: […]. }, you read correctly, this is an object, this object uses a number as a key, a two-dimensional tuple as a value:


  • [0] The first is our wrapped code
  • [1] The second one is oursmapping


We create a top-level require function that takes an ID as a value and returns a brand new Module object. We pour in the module we just created and add {} to it. Make it {1:[…] , 2: […]. } this is a complete form.


And stuff it into our immediate function(function(modules) {… })(), in (function(modules) {… })(), we call require(0) first, simply because our main module is always in first place. Then, in our require function, we get modules passed in from the outside, using the global numeric ID we’ve been talking about, The result of each module is a two-dimensional tuple.


Then, we’re going to make a subrequire, and the reason we’re doing that is because when we use require in a file, we usually require the address, and the top level require parameter is the ID, don’t worry, we used the idMapping here, Find the ID in idMapping by using the address entered by the user require.


Const newModule = {exports: {}}; newModule = {exports: {}}; , run our function fn(childRequire, newModule, newModule.exports); Returns newModule.exports, the module’s exports object.


The logic here isn’t that different from Node.js.


Finally, write a little test


Test code, I have put in the warehouse, want to test students can go to the warehouse to extract.

The annotated code is also in the repository, click on the address

git clone https://github.com/Foveluy/roid.git
npm i
node ./src/_test.js ./example/index.jsCopy the code

The output

this is loader

hello zheng Fang!
welcome to roid, I'm zheng Fang

if you love roid and learnt any thing, please give me a star
https://github.com/Foveluy/roidCopy the code

reference

  1. https://github.com/blackLearning/blackLearning.github.io/issues/23
  2. https://github.com/ronami/minipack