If you like, please follow my blog or subscribe to RSS feed. Previous articles:
- γ Webpack advanced γ Modular design and implementation of front-end runtime
- Use Babel to avoid webPack compilation runtime module dependencies
- Visual display of webpack internal plug-in and hook relationship π
1. The loader asked ten
In the process of learning Webpack Loader, I also read a lot of relevant articles on the Internet and gained a lot. However, most of them only introduce loader configuration mode or Loader writing mode, and the introduction of parameters, API and other details is not clear.
Here are a few “Loader questions” that I had in mind before reading the Loader source code:
- Where is the default configuration for Webpack handled? Is there any default configuration for Loader?
- There is a concept of resolver in webpack, which is used to resolve the real absolute path of module files.
- As we know, in addition to the loader in the config, you can also write the inline loader, so the inline loader and normal config loader order of execution is what?
- The configuration of the
module.rulesHow does it work and implement in Webpack? - How and when does the Loader come into play in the WebPack compilation process?
- Why is loader executed from right to left?
- What exactly happens if a value is returned in a pitch?
- If you’ve written about loader, you’ve probably used it in the Loader Function
thisHere,thisWhat is it, a Webpack instance? - The loader function
this.dataHow is it done? - How to write an asynchronous loader and how to implement loader asynchronization in Webpack?
Maybe you have a similar question. I will restore the design and implementation principle of Loader and answer these doubts by combining loader related source code.
2. General process of loader running
The webpack compilation process is very complex, but the parts involving loader mainly include:
- Default configuration of loader (Webpack)
- Use loaderResolver to resolve the loader module path
- According to the
rule.modulesCreate a RulesSet rule set - Use loader-runner to run loader
The corresponding general process is as follows:
First, the user configuration is merged with the default configuration in Compiler.js, which includes the Loader part.
Webpack then creates two key objects, NormalModuleFactory and ContextModuleFactory, based on the configuration. They are like two class factories through which you can create the corresponding NormalModule and ContextModule. The NormalModule class is the main focus of this article, and WebPack generates an instance of NormalModule for the module files in the source code.
There are a few necessary steps before the factory can create an instance of NormalModule, the most relevant of which is to resolve the Loader path through the Loader resolver.
After an instance of NormalModule is created, the module is built through its.build() method. The first step in building a module is to use the Loader to load and process the module content. The loader-runner library is the loader runner in Webpack.
Finally, output the module content processed by loader to enter the subsequent compilation process.
The above is the general process of loader in Webpack. The following will be combined with the source code for specific analysis, and in the process of reading the source code analysis, will find the answer to the “loader ten questions”.
3. Specific analysis of loader operation
3.1. Webpack default configuration
Q: 1. Where is the default configuration of webpack handled? Is there any default configuration of Loader?
Webpack, like any other tool, works by configuration. As WebPack evolves, its default configuration changes; Some of the best practices from previous releases have moved into the default configuration of WebPack as versions have been upgraded.
The webpack entry file is lib/webpack.js, which will set the configuration options (source code) at compile time according to the configuration file.
options = new WebpackOptionsDefaulter().process(options);
compiler = new Compiler(options.context);
compiler.options = options;
Copy the codeAs you can see, the default configuration is placed in WebpackOptionsDefaulter. Therefore, if you want to see the details of the current WebPack default configuration items, you can do so in this module.
For example, the default value in the module.rules section is []; However, there is also a module.defaultRules configuration item, which is not open to developers but contains the default loader configuration (Source code) :
this.set("module.rules"[]);this.set("module.defaultRules"."make", options => [
{
type: "javascript/auto".resolve: {}}, {test: /\.mjs$/i.type: "javascript/esm".resolve: {
mainFields:
options.target === "web" ||
options.target === "webworker" ||
options.target === "electron-renderer"
? ["browser"."main"]
: ["main"]}}, {test: /\.json$/i.type: "json"
},
{
test: /\.wasm$/i.type: "webassembly/experimental"}]);Copy the codeIt is also worth noting that WebpackOptionsDefaulter inherits from OptionsDefaulter, which is a encapsulated configuration accessor that encapsulates special methods to manipulate configuration objects.
3.2. To createNormalModuleFactory
NormalModule is a mandatory class function in WebPack. Modules in the source code generate corresponding instances of NormalModule during compilation.
NormalModuleFactory is the factory class of NormalModule. It is created in Compiler.js, the controlling class for the basic compilation process of WebPack. The body (hook) flow in the compiler.run() method is as follows:
. The run () in the trigger a series of beforeRun, run hooks, invoked. The compile () method, the first step is to call this. NewCompilationParams create NormalModuleFactory instance ().
newCompilationParams() {
const params = {
normalModuleFactory: this.createNormalModuleFactory(),
contextModuleFactory: this.createContextModuleFactory(),
compilationDependencies: new Set()
};
return params;
}
Copy the code3.3. Resolve the actual absolute path of the Loader
Q: 2. Webpack has a concept of resolver, which is used to resolve the real absolute path of module files.
In NormalModuleFactory, four hooks are involved before an instance of NormalModule is created:
- beforeResolve
- resolve
- factory
- afterResolve
There are two important ones:
- Resolve resolves the path of the loader module (for example, csS-loader).
- Factory is responsible for creating it based on the return value of the Resolve hook
NormalModuleInstance.
The long method registered on the Resolve hook also includes path resolution for the module resource itself. There are two types of resolvers: loaderResolver and normalResolver.
const loaderResolver = this.getResolver("loader");
const normalResolver = this.getResolver("normal", data.resolveOptions);
Copy the codeIn addition to the config file, there are also inline loaders. Therefore, there are two path resolution methods for loader files: inline loader and Loader in the config file. The Resolver hook handles the inline loader first.
3.3.1. inline loader
import Styles from 'style-loader! css-loader? modules! ./styles.css';
Copy the codeThe above is an example of an inline loader. Request is style-loader! css-loader? modules! . / styles. CSS.
First webpack will parse the required loader (source code) from the request:
let elements = requestWithoutMatchResource
.replace(/ ^ -? ! +/."")
.replace(/!!!!! +/g."!")
.split("!");
Copy the codeSo, from style-loader! css-loader? modules! Two loaders can be retrieved in./styles. CSS: style-loader and CSs-loader.
The “parse module’s Loader array” is then executed in parallel with the “parse module itself”, using the neo-Async library.
The neo-Async library is similar to the Async library in that it provides some tools and methods for asynchronous programming, but is faster than the Async library.
The format of the result returned from the parse is roughly as follows:
[
// The first element is a Loader array[{loader:
'/workspace/basic-demo/home/node_modules/html-webpack-plugin/lib/loader.js'.options: undefined}].// The second element is some information about the module itself
{
resourceResolveData: {
context: [Object].path: '/workspace/basic-demo/home/public/index.html'.request: undefined.query: ' '.module: false.file: false.descriptionFilePath: '/workspace/basic-demo/home/package.json'.descriptionFileData: [Object].descriptionFileRoot: '/workspace/basic-demo/home'.relativePath: './public/index.html'.__innerRequest_request: undefined.__innerRequest_relativePath: './public/index.html'.__innerRequest: './public/index.html'
},
resource: '/workspace/basic-demo/home/public/index.html'}]Copy the codeThe first element is all the inline Loaders involved when the module is referenced, containing the absolute path to the Loader file and configuration items.
3.3.2 rainfall distribution on 10-12. Config loader
Q: 3. We know that inline loaders can be written to inline loaders. What is the sequence between inline loader and normal config loader?
In the previous section, Webpack first resolves the absolute path and configuration of the Inline Loader. Loader (source code) ΒΆ Loader (source code) ΒΆ
const result = this.ruleSet.exec({
resource: resourcePath,
realResource: matchResource ! = =undefined
? resource.replace(/ \? . * /."")
: resourcePath,
resourceQuery,
issuer: contextInfo.issuer,
compiler: contextInfo.compiler
});
Copy the codeNormalModuleFactory has a ruleSet attribute that matches the loader required by the module based on its pathname. Click on the RuleSet details here, which I’ll cover in the next section.
Here we pass the source module path to this.ruleset.exec (), and the result returned is the loader in the config matched by the current module. If you are familiar with Webpack configuration, you will know that there is a Enforce field in module.rules. Based on this field, Webpack will classify loaders into preLoader, postLoader, and Loader (source code) :
for (const r of result) {
if (r.type === "use") {
/ / post type
if (r.enforce === "post" && !noPrePostAutoLoaders) {
useLoadersPost.push(r.value);
/ / the pre type
} else if (
r.enforce === "pre"&&! noPreAutoLoaders && ! noPrePostAutoLoaders ) { useLoadersPre.push(r.value); }else if(! r.enforce && ! noAutoLoaders && ! noPrePostAutoLoaders ) { useLoaders.push(r.value); }}/ /...
}
Copy the codeFinally, use neo-aySNc to parse three loader arrays in parallel (source code) :
asyncLib.parallel(
[
this.resolveRequestArray.bind(
this,
contextInfo,
this.context,
useLoadersPost, // postLoader
loaderResolver
),
this.resolveRequestArray.bind(
this,
contextInfo,
this.context,
useLoaders, // loader
loaderResolver
),
this.resolveRequestArray.bind(
this,
contextInfo,
this.context,
useLoadersPre, // preLoader
loaderResolver
)
]
/ /...
}
Copy the codeSo what is the final loader order? The following line of code can explain:
loaders = results[0].concat(loaders, results[1], results[2]);
Copy the codeResults [0], Results [1], Results [2] and Loader are postLoader, Loader (Normal Config Loader), preLoader and inlineLoader respectively. Therefore, the combined loader sequence is POST, inline, Normal, and pre.
However, loader is executed from right to left. The actual loader execution sequence is reversed. Therefore, inlineLoader is executed after normal Loader in config.
3.3.3. RuleSet
Q: 4. How to implement the module.rules in webpack?
Webpack uses RuleSet objects to match the loader required by the module. RuleSet is a rule filter that applies resourcePath to all module.rules rules to filter out the required loaders. Two of the most important are:
- Class static method
.normalizeRule() - Instance methods
.exec()
Webpack compilation instantiates a RuleSet based on user configuration and default configuration. First, a static method on it. NormalizeRule () converts the configuration value to a standardized test object; A this. References property is stored on the map, and the key is the type and location of the loader in the configuration. For example, ref-2 represents the third loader configuration array.
P.S. If you print request fields from the NormalModule on a. Compilation hook, those modules that use the Loader will have ref-like values. From here, you can see whether a module uses loader and which configuration rules it matches.
The instantiated RuleSet can then be used to get the corresponding Loader for each module. The instantiated RuleSet is the this.ruleset property on the NormalModuleFactory instance we mentioned above. The factory calls the RuleSet instance’s.exec() method each time it creates a new NormalModule, and pushes the Loader into the result array only if the various test conditions pass.
3.4. Run the loader
3.4.1. Operation timing of Loader
Q: 5. How and when does loader come into play in the Webpack compilation process?
After the absolute path of the Loader is resolved, a NormalModule object for the current module is created in the NormalModuleFactory Factory hook. So far, the loader preprocessing work is almost finished, the following is to actually run each loader.
As we all know, running loader read and process module is the first step of webPack module processing. But when it comes to runtime timing, the compilation is a very important object in webPack compilation.
Webpack is compiled in the entry dimension, and one of the important methods in compilation,.addEntry(), builds modules based on the entry dimension. The._addModulechain () call in the.addEntry() method will execute a series of module methods (source code)
this.semaphore.acquire((a)= > {
moduleFactory.create(
{
/ /...
},
(err, module) = > {if (err) {
this.semaphore.release();
return errorAndCallback(new EntryModuleNotFoundError(err));
}
/ /...
if (addModuleResult.build) {
// Module building
this.buildModule(module.false.null.null, err => {
if (err) {
this.semaphore.release();
return errorAndCallback(err);
}
if (currentProfile) {
const afterBuilding = Date.now();
currentProfile.building = afterBuilding - afterFactory;
}
this.semaphore.release(); afterBuild(); }); }})}Copy the codeFor unbuilt modules, the.dobuild () method of the NormalModule object is eventually called. The first step in building the module (.dobuild ()) is to run all the Loaders.
That’s where the Loader-runner comes in.
Loader -runner — An execution library for loader
Q: 6. Why is loader executed from right to left?
Webpack separates loader’s running tools into loader- Runner libraries. Therefore, you can write a loader and use a separate Loader-runner to test the loader.
Loader-runner is divided into two parts: loadloader.js and LoaderRunner.
Loadloader.js is a compatible module loader that can load module definitions such as CJS, ESM, or SystemJS. LoaderRunner. Js is the core part of loader module. The exposed.runloaders () method is used to start the loader.
If you have written or know how to write a Loader, you know that every Loader module supports a.pitch property, which takes precedence over the actual method execution of the Loader. In fact, the webpack official also gives the pitch and loader’s own method execution sequence diagram:
|- a-loader `pitch` |- b-loader `pitch` |- c-loader `pitch` |- requested module is picked up as a dependency |- c-loader normal execution |- b-loader normal execution |- a-loader normal executionCopy the codeThe two phases (pitch and normal) are the corresponding iteratePitchingLoaders() and iterateNormalLoaders() methods in loader-Runner.
IteratePitchingLoaders () executes recursively and records the pitch state of the loader and the loaderIndex (loaderIndex++) currently executed. The actual Module is processed only when the maximum loader sequence number is reached:
if(loaderContext.loaderIndex >= loaderContext.loaders.length)
return processResource(options, loaderContext, callback);
Copy the codeWhen loaderContext. LoaderIndex value to achieve the overall loader array length, suggests that all pitch have been completed (perform the final loader), then calls the processResource () resources to deal with the module. It mainly includes adding the module as a dependency and reading the contents of the module. IterateNormalLoaders () is then recursively executed and the loaderIndex– operation is performed, so the Loader executes “backwards”.
Next, let’s discuss a few loader-runner details:
Q: 7. What exactly happens if a value is returned in one pitch?
The website says:
if a loader delivers a result in the pitch method the process turns around and skips the remaining loaders
This description indicates that the return value in pitch skips the rest of the loader. This is a rough statement with a few details:
First, processResource() is executed only when loaderIndex reaches the maximum array length, i.e. pitch has passed all loaders.
if(loaderContext.loaderIndex >= loaderContext.loaders.length)
return processResource(options, loaderContext, callback);
Copy the codeTherefore, in addition to skipping the rest of the loader, returning a value in the pitch not only makes.adddependency () untriggered (the module resource is not added to the dependency), but the module’s file contents cannot be read. Loader will process the value returned by pitch as “file content” and return it to Webpack.
Q: 8. If you have written loader, you may use this in loader function.
This is not an instance of WebPack, nor is it an instance of Compiler, Compilation, normalModule, etc. It’s a Loader-Runner specific object called loaderContext.
Each time the runLoaders() method is called, a new loaderContext is created by default if the context is not explicitly passed. So the various Loader apis mentioned on the official website (callback, data, loaderIndex, addContextDependency, etc.) are attributes on this object.
Q: 9. How to implement this.data in loader function?
Loadercontext.data loaderContext.data loaderContext.data loaderContext.data loaderContext.data loaderContext.data loaderContext.data
Object.defineProperty(loaderContext, "data", {
enumerable: true.get: function() {
returnloaderContext.loaders[loaderContext.loaderIndex].data; }});Copy the codeSo we’ve defined a fetch of.data. When this. Data is called, different normal loaders will get different loaderIndex values. The parameter data of the pitch method is also data (source code) of different loaders.
runSyncOrAsync(
fn,
loaderContext,
[loaderContext.remainingRequest, loaderContext.previousRequest, currentLoaderObject.data = {}],
function(err) {
/ /...});Copy the codeRunSyncOrAsync () arrays in [loaderContext remainingRequest, loaderContext. PreviousRequest, Currentloaderobject. data = {}] is the input parameter to the pitch method, and currentLoaderObject is the loader object to which the current loaderIndex refers.
Therefore, if you want to store “through-through” data, consider storing it on another property of this, or changing the loaderIndex to fetch data from another loader (compare hacks).
Q: 10. How to write an asynchronous loader and how to implement loader asynchronization by Webpack?
The actual execution of the pitch and normal loader is in the runSyncOrAsync() method.
According to the Webpack documentation, when we call this.async(), it turns the Loader into an asynchronous loader and returns an asynchronous callback.
Implementation-specific, runSyncOrAsync() contains an isSync variable, which defaults to true; When we call this.async(), it is set to false and returns an innerCallback as a callback notification when the async execution is complete:
context.async = function async() {
if(isDone) {
if(reportedError) return; // ignore
throw new Error("async(): The callback was already called.");
}
isSync = false;
return innerCallback;
};
Copy the codeWe usually use the callback returned by this.async() to notify asynchronous completion, but in practice, this.callback() does the same thing:
var innerCallback = context.callback = function() {
/ /...
}
Copy the codeAlso, in runSyncOrAsync(), only if isSync flag is true will the (synchronous) callback continue loader-runner immediately after the loader function completes execution.
if(isSync) {
isDone = true;
if(result === undefined)
return callback();
if(result && typeof result === "object" && typeof result.then === "function") {
return result.catch(callback).then(function(r) {
callback(null, r);
});
}
return callback(null, result);
}
Copy the codeAs you can see here, there is a place in the code that determines if the returned value is a Promise (typeof result.then === “function”), and if it is a Promise then callback will be called asynchronously. Therefore, to get an asynchronous loader, you can return a Promise directly, in addition to the this.async() method mentioned in the Webpack documentation.
End of 4.
The above is webAPck Loader related part of the source code analysis. At this point, you have the answer to the first “Loader 10 Questions”. I hope this article will give you a further understanding of loader implementation in addition to learning how to configure loader and write a simple Loader.
There may be some flaws in the process of reading the source code, welcome to communicate with you.
Farewell to webPack Configuration Engineer
Webpack is a powerful and complex front-end automation tool. One of these features is the complexity of configuration, which has led to the popularity of the tongue-in-mouth term “WebPack configuration engineer” π€· But are you really content with just playing around with WebPack configuration?
Apparently not. In addition to learning how to use WebPack, we need to explore the design and implementation of various parts of Webpack. Even when WebPack is “out of fashion,” some of its designs and implementations will still be worth learning from. Therefore, in the process of learning Webpack, I will summarize a series of [Webpack advanced] articles and share with you.
Welcome interested students more exchanges and attention!
Previous articles:
- γ Webpack advanced γ Modular design and implementation of front-end runtime
- Use Babel to avoid webPack compilation runtime module dependencies
- Visual display of webpack internal plug-in and hook relationship π