preface

In the last article, we implemented a simple packer and briefly looked at how WebPack works. To verify our guess, let’s go to the source code

How to read the source code

Have the source code ready on your home page before reading it (duh)

There are two ways to get the source code

The first method is Git Clone, which can get various versions of the Webpack code. When the Webpack is updated, you can also pull the latest code by Git pull. However, due to network reasons, I have been unable to pull it down with Git Clone. I chose the second option

The second option is to download through Releases, which requires selecting the corresponding version to download. In this case, I selected version 5.10.0 and downloaded the corresponding compressed file

Read the source code with questions

Webpack code volume is huge, there are many branches, complex, if you want to read the code line by line, it is obviously not feasible, is also inefficient, so our goal should be to only look at the core code, understand the overall process of Webpack packaging

In order to understand the overall running process of Webpack as a big goal, we need to break it down into small goals or problems, and understand the whole context in the process of finding answers by constantly disassembling the problems

So the first question is: where does compilation start?

The source code parsing

How do I run WebPack

Before we get started, there are two ways to start webPack

The first method uses webpack-CLI to launch from the command line

npx webpack-cli
Copy the code

This is the fastest and easiest way to do it, and the default entry for packaging is SRC /index.js

The second method is implemented by requiring (‘webpack’) importing the package

Both methods call WebPack to package the code, just in different ways

Problem 1: compilation starting point

Everything from compiler = webpack(options, callback); start

Webpack function source code (lib/webpack.js)

const webpack = ((options,callback) = > {
    const create = () = >{...let compiler;
        if (Array.isArray(options)) {
            ...
        } else {
            / * *@type {Compiler} * /compiler = createCompiler(options); . }return { compiler, watch, watchOptions }
    };
    if (callback) {
        try {
            const { compiler, watch, watchOptions } = create();
                if (watch) {
                    compiler.watch(watchOptions, callback);
		} else {
                    compiler.run((err, stats) = > {
			compiler.close(err2= > {
                            callback(err || err2, stats);
			});
                    });
		}
		return compiler;
            } catch (err) {...}
	} else {...}
});
Copy the code

Compiler. Run is the core code, representing the beginning of compilation, while createCompiler mainly instantiates compiler and does some initialization, such as reading configuration, activating the built-in plug-in of Webpack…

In Webpack, Compiler is one of the core objects. Through the instantiation of new Compiler, Compiler records the complete information of Webpack environment. From the start of Webpack to the end, Compiler will be generated only once. You can read webpack Config information, outputPath, etc

fromcomilper.run()The compilation really started

Tapable

When reading the webpack source code, you will often see xxx.hooks. XXX, which is actually an event management library called Tapable, and to continue reading the source code you must quickly learn the basic usage of Tapable

// Define an event/hook
this.hooks.eventname = new SyncHook(['arg1'.'arg2'])
// Listen for an event/hook
this.hooks.eventName.tab('Reason for monitoring', fn)// Triggers an event/hook
this.hooks.eventName.call('arg1', arg2)
Copy the code

Question 2: What is the process of Webpack

Compiler. Run () is the start of compiling

Find the Compiler class definition file lib/Compiler.js (source)

run(callback) {
    const onCompiled = (err, compilation) = >{... };const run = () = > {
        this.hooks.beforeRun.callAsync(this.err= >{...this.hooks.run.callAsync(this.err= >{...this.readRecords(err= >{...this.compile(onCompiled);
               });
            });
          });
    };

    if (this.idle) {
        ...
	run();
    });
    } else{ run(); }}Copy the code

In compiler.run code, we can see that hooks such as hooks. BeforeRun hooks. Third party plug-ins can define handlers for different events that fire at the corresponding compile time to achieve plug-in effects

In the process of reading, we should write down important hooks and methods for easy reading

For example, the code above can be written down as

Run-beforerun -run compiler.readRecords compiler. piler...Copy the code

In this way, we can quickly understand the webpack process

And then from above, we’re going to look at compiler.compiler

Lib/webpack source code Compiler. Js

compile(callback) {
    const params = this.newCompilationParams();
    this.hooks.beforeCompile.callAsync(params, err= > {
        this.hooks.compile.call(params);
        const compilation = this.newCompilation(params);
        this.hooks.make.callAsync(compilation, err= > {
            this.hooks.finishMake.callAsync(compilation, err= > {
                process.nextTick(() = > {
                    compilation.finish(err= > {
                            compilation.seal(err= > {
				this.hooks.afterCompile.callAsync(compilation, err= > {
                                    return callback(null, compilation);
                                });
                            });
                        });
                    });
                });
            });
        });
}
Copy the code

Because the source details are more, so the above brief some error handling and log information

Continue documenting the compilation process

Run -beforeRun -run compiler.readRecords compiler. piler -beforeCompile -compile newCompilation -make -finishMake nextTick compilation.finish compilation.seal -afterCompileCopy the code

Env > init > run > compile > compilation > make > finishMake > afterCompile > seal > emit These stages

In addition, we can see a heavyweight object compilation in the code, as can be seen from the name, this object must play an important role in the compilation process. Compilation represents a single compilation job. The compilation records the module resources of the current compilation job and the files generated by the compilation, as well as dependency information.

Problem 3: read index.js and analyze the dependency at what stage

Following the stages we have just analyzed, we can first exclude env init (presumably), certainly between Compiler > afterCompile

The make > finishMake phases are the most likely

Why is that? If you know C, make is a necessary tool for compiling

So let’s look at these two stages

Paste source code directly

This code does nothing except logger and error handling. Why did it change from make to finishMake?

What is done between make-finishmake

Remember the Tabable library mentioned earlier? This is webPack’s event distribution system, so we can start with the make event binding handler

A direct global search for make. TapAsync shows that the plug-in is defining the handler

EntryPlupin is a must-look for dependencies that need to be analyzed before they are compiled

If you look at the make. TapAsync (“EntryPlugin”) code, It actually calls the compilation.addentry, After multiple lookups compilation. AddEntry – > compilation. _addEntryItem – > compilation. AddModuleChain – > compilation.handleModuleCreation

Let’s look at handleModuleCreation

handleModuleCreation({ factory, dependencies, originModule, context, recursive = true },callback){...this.factorizeModule({ currentProfile, factory, dependencies, originModule, context },(err, newModule) = >{...this.addModule(newModule, (err, module) = >{...this.processModuleDependencies(module.err= >{... callback(null.module); }); }); }); }Copy the code

FactorizeModule (). Click on this function and we’ll finally see this.factorizequeue.add (options, callback).

So what exactly is factorizeQueue? You can look at his definition

In this._factorizemodule, we end up with factory.create().

Question 5: What is factory.create

Here we need to touch the parameter according to invoke relationship up, looking for, in the end we found clues in the addModuleChain, factory from this. DependencyFactories. Get (Dep)

Then clues interrupted this. DependencyFactories. Get (Dep)? Given the get it should have the set, through global search again (luck), finally found in EntryPlugin

The factory is normalModuleFactory

So the factory. The create is normalModuleFactory. Create

Because normalModuleFactory is too long, NMF for short

Stage summary

So far we know that

  • webpackusehooksNail down the main stages
  • webpackDuring execution, the plug-in does things at its chosen stage
  • Entry is made by entry plug-inEntryPlugin.jsFix the

After you clear your head, move on…

Q6. What does nMF.Create do

Source the NMF. Create

create(data, callback){...this.hooks.beforeResolve.callAsync(resolveData, (err, result) = >{...this.hooks.factorize.callAsync(resolveData, (err, module) = >{... callback(null, factoryResult);
        });
    });
}
Copy the code

Here we can see that two hooks,boforeResolve and factorize, are actually triggered. By looking at the corresponding event handlers, we can see that factory.tap contains important code

It triggers Resolve, whose main purpose is to collect loaders

It then triggers the createModule to get the createdMolude

That is, factory.create collects loaders and gets the Module object

Q7: What does addModule do

Now that we have clear factory. What did the create, then we return to compilation. FactorizeModle () (forget the four code chart) can be back to see a problem, found the back of the operation is addModule and buildModule

So what does addModule do?

AddModule adds a module to compilation. Modules

Looking back at the code in q4, we can see that the addModule phase actually adds the newModule from factory.create() to copilation. Modules

What does buildModule do

Look at the name, it’s an important step

BuildModule actually calls normalModule.build, which calls its own doBuild method

const { runLoaders } = require("loader-runner");
doBuild(options, compilation, resolver, fs, callback){
    // runLoaders method imported from package 'loader-runner'
    runLoaders({
        resource: this.resource,  // The resource may be a js file, a CSS file, or an img file
        loaders: this.loaders,
    }, (err, result) = > {
        const source = result[0];
        const sourceMap = result.length >= 1 ? result[1] : null;
        const extraInfo = result.length >= 2 ? result[2] : null;
        // ...})}Copy the code

The runLoaders method is used to load non-JS files, such as CSS, HTML… , to js (Webpack only recognizes JS), runLoaders gets result to proceed to the next step

The next step is to convert JS into AST code

result = this.parser.parse(source);

This.parse. Parse is actually a parse method provided by the third-party library Acorn

parse(code, options){
    // Call the third-party plugin 'Acorn' to parse the JS module
    let ast = acorn.parse(code)
    ...
    if (this.hooks.program.call(ast, comments) === undefined) {
        this.detectStrictMode(ast.body)
        this.prewalkStatements(ast.body)
        this.blockPrewalkStatements(ast.body) // Analyze dependencies
        this.walkStatements(ast.body)
    }
}
Copy the code

How does Webpack know which files index.js depends on

In the last question, we have analyzed that WebPack uses runLoaders to convert code to JS, and then converts JS to AST code via the Acorn library

So how does WebPack analyze dependencies?

According to the principle of the previous article, we should traverse the AST and look for import statements

In the source code can be found in the enclosing blockPrewalkStatements ImportDeclaration (ast) body) on the inspection, once found the import ‘XXX’, will trigger the import hooks, corresponding monitoring function handles dependencies, Add dependencies to the array of module.dependencies

Question 10: How to combine modules into a file

Env > compile > make > seal > emit

We can guess that the merge file will be in between the SEAL > EMIT phase

At this stage, compilation.seal is called, which creates chunks, cobugs each chunk, and then creates an asset for each chunk

After SEAL comes the EMIT phase, which is the emit phase, and the file is obviously written out, resulting in dist/main.js and other chunk files

conclusion

This article is quite long, thank you very much for reading to the end, and finally briefly summarize the basic flow of Webpack:

  1. Env and init phases, calledwebpackThe function to receiveconfigConfigure information andapplyallwebpackBuilt-in plug-ins
  2. callcompiler.runEnter the compile phase
  3. The make phase, fromentryFor the entrance, throughloadersConvert the source code of the module to JS module, and then useacornParse into an AST (abstract syntax number), which iterates the syntax tree to analyze and collect dependencies
  4. Seal stage,webpack 将 moduletochunk, eachchunkIt can be one module or multiple modules
  5. Emit phase, for eachchunkCreate a file and write it to hard disk

Code word is not easy, if this article is useful to you please give it a thumbs up

If there is a bad place to write, welcome the leaders to guide