Loader principle and implementation of Webpack

Webpack loader principle

In the last article, we looked at the implementation of WebPack. The packaging analysis of WebPack 5. In this article, we will look at how WebPack handles the content of non-JS files. This series is divided into three parts

1. Webpack5 packaging analysis
2. Webpack loader implementation
3. Webpack’s event flow and plugin principles

The principle of the Loader

We know that inside WebPack there is a concept of everything as a module, but what if we deal with content that is not a JS file?

/* Webpack-loader-study/SRC ├── SRC ├─ index.js ├─ index.css ├─ public ├─ index.html */
// index.js
import "./index.css"
console.log('hello world')
Copy the code

Our WebPack configuration is much simpler

const path = require('path')
const HtmlWebpackPlugin = require('html-webpack-plugin')
module.exports = {
  mode: 'development'.entry: "./src/index.js".output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'build.js'
  },
  plugins: [
      new HtmlWebpackPlugin({
        template: './public/index.html'}})]Copy the code

When we run NPX webpack, we will find an error and find that webpack does not know our CSS file, so this will require a certain loader to process the content of the file that is not JS

ERROR in ./src/index.css 1:5 Module parse failed: Unexpected token (1:5) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file.

In this example, we can use CSS-loader to make our webpack process CSS files. After we do this, we find that the CSS file does not return errors, but it does not work because csS-Loader simply converts the CSS file to a string. No other logic is done, so we still need to extract strings into style or separate files

The module: {rules: [{$/ test: / \. CSS, loader: 'CSS - loader'}]} # # modified into+ module: {
+ rules: [
+ {
+ test: /\.css$/,
+ use: ['style-loader', 'css-loader']
+}
+]
+}
Copy the code

Development of custom loader

1. Loader is a function that takes a source code as an argument
2. Loader cannot be an arrow function because arguments need to be passed through this context
3. The return value must be a buffer or string. Asynchronously, specific functions need to be called

// The synchronized loader returns the content directly
module.exports = function (source) {
  console.log(source)
  return source.replace('hello'.'hcc')}// The loader that implements synchronization through this.callback can add some parameters (error message, resource, sourcemap)
module.exports = function (source) {
  console.log(source)
  let newSource = source.replace('hello'.'hcc-1-2')
  return this.callback(null, newSource)
}
Copy the code

If there are some asynchronous operations to replace the source code, then if we do this directly, we will get an error because it says that the loader has finished executing and no value is returned.

module.exports = function (source) {
  let newSource = source.replace('hello'.'hcc-1-2')
  setTimeout(() = > {
    return this.callback(null, newSource)
  }, 2000)}Copy the code

Therefore, we need to let it know that our processing is not complete. We can use the callback returned by this.async to actively notify loader whether the processing is complete

module.exports = function (source) {
  let callback = this.async()
  let newSource = source.replace('hello'.'hcc-2-2')
  setTimeout(() = > {
    return callback(null, newSource)
  }, 2000)}Copy the code

Handwriting commonly used loader

Above we can roughly know the purpose and use of loader, next we will write some loader to process some files, convenient for you to deepen the understanding of Loader

1. Based on the jsbabel-loader
2. Based on pictures and documentsfile-loader.url-loader
3. style-basedstyle-loader.css-loader.less-loader

Js based compatible processing

We know that low end browsers have compatibility issues when we write high end syntaxes like ES7, so we need to use babel-Loader to convert high end syntaxes to low end syntaxes

// in index.js we write a class
class A {
  getName() {
    console.log('name')}}Copy the code

Next, we will implement our own babel-loader. When we use babel-loader, we will execute the following sentence directly, but there are many questions

1. Why didn’t I use it@babel/coreBut what about installing dependencies as well?
2. @babel/preset-envWhat is the function of “options” and why is it used in options

npm install -D babel-loader @babel/core @babel/preset-env
Copy the code

Original effect

1. inwebpack.config.jsConfigure loader to process JS files

+ {
+ test: /\.jsx? $/,
+ use: {
+ loader: 'babel-loader',
+ options: {
+ presets: [
+ '@babel/preset-env'
+]
+}
+}
+}
Copy the code

When we run NPX webpack and look at the output, we find that the class has been escaped as follows

function A() {
    _classCallCheck(this, A);
}
Copy the code

implementationbabel-loader

1. Let’s create one in the loader directoryhcc-babel-loader.jsfile
2. inwebpack.config.jsThe same configuration for loader to process JS files

{ test: /\.jsx? $/, use: { _ loader: 'babel-loader',+ loader: 'hcc-babel-loader',
          options: {
            presets: [
                '@babel/preset-env'
            ]
          }
        }
      }
Copy the code

3. inhcc-babel-laoderThrough the core Babel module (@babel/core) to deal with the code

let babel = require('@babel/core')
let loaderUtils = require('loader-utils')
module.exports = function(source) {
  let cb = this.async()
  let options = loaderUtils.getOptions(this) // Get the options passed in
  babel.transform(source, { // babel.transform is an asynchronous operation. options, },(err, result) = > {
    cb(err, result.code)
  })
}
Copy the code

4. This packaged code is Babel escaped code, but we packaged it withoutsource-mapConfiguration, need to deal with can output source map, againwebpack.config.jsNeed to opendevtool: source-map

// let Babel = require('@babel/core') let loaderUtils = require('loader-utils') module. Exports = // let Babel = require('@babel/core') let loaderUtils = require('loader-utils') module Function (source) {let cb = this.async() let options = loaderutils.getoptions (this Babel.transform (source, {// babel.transform is an asynchronous operation... options,+ sourceMaps: true
  }, (err, result) => {
+ cb(err, result.code, result.map)})}Copy the code

Implement file-loader and URl-loader

Added image processing to webpack.config.js

{ test: /\.(png|jpe? g|gif)(\? . *)? $/, loader: 'hcc-file-loader' }Copy the code

Add the following code to the entry file

import pic from './pic/hcc.jpg'
let image = new Image()
image.src = pic
document.body.appendChild(image)
Copy the code

Before writing file-loader and url-loader, a few concepts need to be explained

1. url-loaderincludingfile-loaderImages must be converted to binary before they can be processed **loader.raw = true;* *.
2. file-laoderThe principle is that whenwebpackWhen we find that we are dealing with images and other suffixes, we process the content of the image and export a packaged address. When we import the image, we import the packaged address
3. Let’s create a new onehcc-file-loaderThe file,

const loaderUtils = require('loader-utils')
function loader(source) {
  let filename = loaderUtils.interpolateName(this.'[hash].[ext]', {
    content:source
  })
  console.log(filename)
  this.emitFile(filename, source) // Send files to the dist directory
  return `module.exports = "${filename}"`
}
// Convert the image to binary
loader.raw = true;
module.exports = loader
Copy the code

Here we need to export the file (module.export =…) Because there is a PIC variable in index.js that needs to be imported into the converted content, this can cause problems if you are not careful.

We have handled file-loader above, next we will implement url-loader based on file-loader

1. Added the configuration of url-loader

      {
        test: /\.(png|jpe? g|gif)(\? . *)? $/,
        use: {
          loader: 'url-loader'.options: {
            limit: 20000}}}Copy the code

2. To create ahcc-url-loaderFor file size processing, if the file is greater than 2K usefile-loaderCreate a separate file and convert it to base64 if it is less than 20K

const loaderUtils = require('loader-utils')
const mime = require('mime');
function loader(source) {
  const { limit } = loaderUtils.getOptions(this);
  console.log(source.length)
  let size = source.length // Get the file size
  // Use file-laoder to process files
  if(limit && size > limit) {
    return require('./hcc-file-loader').call(this, source) // Pay attention to the passing context
  } else {
    / / return base64
    return `module.exports = "data:${mime.getType(this.resourcePath)}; base64,${source.toString('base64')}"`;
  }
}
loader.raw = true
module.exports = loader
Copy the code

Processing style

Based on the style processing is divided into the following three loaderstyle-loader, CSS-loader, less-loader

1. A LESS file passes through firstless-loaderTo convert the LESS syntax into CSS syntax
2. Then throughcss-loaderFor CSS processing, the use of images in THE CSS needs to passurl-loaderTo deal with
3. throughstyle-loaderInsert into the template’s style tag

implementation

1. We are nowwebpack.config.jsCreate a corresponding loader to process less files

  {
    test: /\.less$/,
    use: {
      loader: ['hcc-style-loader', 'hcc-css-loader', 'hcc-less-loader']
    }
  }
Copy the code

2. To create aindex.lessFile and then file in the entryindex.jsThe introduction of

body {
  background: red;
  #app{
    color: skyblue; }}Copy the code

Index. Js file

import './index.less'
Copy the code

3. hcc-less-loaderTo use less’s render to convert less content to CSS content, so we used it beforeless-loaderNeed to install less dependencies becauseless-loaderIt is used to transform syntax in

let less = require('less')
module.exports = function(source) {
  let result = ""
  less.render(source, null.(err, output) = > {
    result = output.css
  })
  return result
}
Copy the code

4. hcc-css-loaderTo output the CSS, we’ll do a little processing here, and we’ll talk about the problem in more detail later

module.exports = function(source) {
  return source
}
Copy the code

5. hcc-style-loaderTo store the CSS in the newstyleIn the label

module.exports = function (source) {
  return `let style = document.createElement('style')
    style.innerHTML = The ${JSON.stringify(source)}/ / note that formatting code, a newline keep the document. The head. The appendChild ` (style)
}
Copy the code

The problem

If we add the contents of the image in index.less, there will be a problem that the image cannot be loaded. Because CSS-loader does not do any processing, when it is passed to style-loader, it cannot find the corresponding image location in dist directory after packaging

body {
  background: url("./pic/hcc.jpg");
  #app{
    color: skyblue; }}Copy the code

Therefore, we need to modify the CSS-loader. We need to implement the image address of the above URL through require, so that when packaging, it will passurl-loaderTo process the content of the image and send it to the dist directory

body {
 background: url(require('./pic/hcc.jpg'))}body #app {
 color: skyblue;
}
Copy the code

But we changed it this way, there is another problem, require needs to be executed at runtime, but style-loader is executed at compile time, so there is a problem, csS-loader after the value is not used in style-loader, we need some extra processing here. First we need to understand the loader execution mechanism

Loader execution mechanism

1. If we write like thisuse: ['loader1', 'loader2', 'loader3']Under normal circumstances, the real loader is executed in this way, and it will enter the Loader firstpitch, and then go backwards, but we usually omit itpitchLoader executes from left to right

2. When we writepitchThe loader execution mechanism will change ifpitchIf there is a return value, it will skip subsequent execution and jump directly to the previous normal loader, and the loader chain will break. For example,loader2We used inloader2.pitchAnd returns the content, which becomes the following executionWith this knowledge, we can introduce it dynamicallycss-loaderTo implement the

Final implementation

1. instyle-loaderAdd apitch, skip subsequent loader processing
   • pitchThere is an argument that has a surplus that has not been executedloaderAnd source files
   • inpitchAfter processing, skip all loaders and run the file directly.!!!!!Skip pre, Normal, and Post loader)
   • require()You need a relative path, you need to go throughloader-utilsTo process

let loaderUtils = require('loader-utils')
module.exports = function (source) {
  console.log('normal-style-loader', source)
  return ""
}
module.exports.pitch = function(remainingRequest) {
  // E:\hcc\hcc-webpack\webpack-loader-study\loader\hcc-css-loader.js! E:\hcc\hcc-webpack\webpack-loader-study\loader\hcc-less-loader.js! E:\hcc\hcc-webpack\webpack-loader-study\src\index.less
  console.log(remainingRequest)
  console.log('pitch-style-loader')
  / / pass!!!!! Skip the rest of the loader, equivalent to executing require('index.less')
  return `
    let style = document.createElement('style')
    style.innerHTML = require(${loaderUtils.stringifyRequest(this.'!!!!! ' + remainingRequest)})
    document.head.appendChild(style)`
}
Copy the code

2. css-loaderRequire splicing of images in

module.exports = function(source) {
  console.log('css-normal-loader', source)
  let arr = ['let list = []']
  let pos = 0
  let reg = /url\((.+?) \)/g
  let current;
  while(current = reg.exec(source)) {
    let [matchUrl, group] = current
    // console.log(matchUrl, group)
    let last = reg.lastIndex - matchUrl.length;
    arr.push(`list.push(The ${JSON.stringify(source.slice(pos, last))}) `)
    pos = reg.lastIndex
    // Notice the string concatenation here
    arr.push(`list.push('url(' + require(${group}`) + ') '))
  }
  arr.push(`list.push(The ${JSON.stringify(source.slice(pos))}) `)
  arr.push(`module.exports = list.join('')`)
  console.log(arr.join('\r\n'))
  return arr.join('\r\n')}Copy the code

3. less-loaderIt doesn’t have to change anything. It’s the same thing as before

let less = require('less')
module.exports = function(source) {
  let result = ""
  less.render(source, null.(err, output) = > {
    result = output.css
  })
  return result
}
Copy the code

4. The modified run path looks like this
   • Executed firsthcc-style-loaderthepitchAnd then executerequire(index.less)Enter less file processing in Webpack
   • css-loaderExports required modules through module.exports
   • style-loadrAdd to innerHTML of the style tag, and then render to the page style