The preface

Build tools are essential to the development of modern front-end projects. Whether your project is old or new, you’ve probably been exposed to some of these tools, such as WebPack /Rollup, which is currently the dominant packaged build tool. It could also be a newer bundless build tool like Vite/Snowpack, or a more traditional build tool like Grunt/Gulp/Browserify. However, rather than delving directly into how these dizzying build tools are configured, we need to think about:

  1. Why do front-end projects need to be built?
  2. Why are there so many build tools?
  3. Which build tool should I use for my project?

If you have any questions about any of the above, take a few minutes to read this article.

What is building?

First, let’s define a build:

Build is all about converting the code we write during development into the code we deploy in production.

Admittedly, this definition is not the most accurate, but it is certainly the most simple and clear. Yes, there are a variety of build tools out there, but they all have the same ultimate goal: to convert code from development to production. Around this end goal, different build tools add and focus on different features as selling points. (file packaging, code compression, Code splitting, tree shaking, Hot Module replacement, etc.)

It is important to understand, however, that the number of these features is not the only judgment when choosing a build tool, nor is it the main reason for the build tool’s new iterations. To understand why build tools are being updated, how they got where they are, and where they might go in the future, we need to start with the modularity of the front end.

Two, slash-and-burn – JS inline and outreach

Is front-end build required? Of course not!

I don’t know how many of you remember when you first learned about the front end, you just had to format a few HTML tags, then insert a simple JS code, open the browser Hellp World and display it on our interface.

<html>
  <head>
    <title>Hello World</title>
  </head>
  <body>
    <div id="root"/>
    <script type="text/javascript">
      document.getElementById('root').innerText = 'Hello World'
    </script>
  </body>
</html>
Copy the code

As you can see, the inline JS code runs successfully in the browser without being processed by any build tools. Such code might be maintainable on a small learning project with no more than a few hundred lines of code, but when the project gets into real development and the code scales up rapidly, the amount of logic mixed up in a single file becomes unbearable. Therefore, most of the front-end project code is organized as follows.

<html>
  <head>
    <title>JQuery</title>
  </head>
  <body>
    <div id="root"/>
    <script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
    <script type="text/javascript">
      $(document).ready(function(){$('#root') [0].innerText = 'Hello World'
      })
    </script>
  </body>
</html>
Copy the code

This external reference to JS allows the code logic to be split into files. For example, JQuery code will be placed in JQuery files, and our project code can be placed in files created by ourselves.

However, external reference JS is just a cover for the messy problem of code organization: the large number of global variables in our projects and the dependence of code references on a specific order make code maintenance more complex.

<html>
  <head>
    <title>JQuery</title>
  </head>
  <body>
    <div id="root"/>
    <script type="text/javascript">
    	// undefined!! At this point, JQuery is not loaded and the $variable is not defined
      console.log($)
    </script>
    <script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
  </body>
</html>
Copy the code

Since then, methods such as IIFE and namespaces have emerged to control the complexity of code, but these methods do not inherently solve the problem of relying on global variable communication between codes. In this context, JS modularization has become the only option for the front end to be on the right track of engineering. However, due to the inherent shortcomings of JS, modularity does not exist in the original design of JS, so different modularity methods began to emerge gradually.

AMD/ CMD-asynchronous module loading

In order to solve the problem of browser-side JS modularization, the first accepted way is to introduce the relevant tool library to achieve. Among them, the specification AMD(Asynchronous Module Definition) proposed and promoted by RequireJs was the most widely adopted solution at that time.

define(id? , dependencies? , factory);

define("mycode"["jquery"].function ($) {$(document).ready(function(){$('#root') [0].innerText = 'Hello World'
  })
  return$})Copy the code

AMD specification adopts the method of dependency preloading, which puts all dependencies into the Dependecies array. RequireJS will first load and execute the files passed in the Dependecies array during execution, and then take the return value of these files as the parameter of factory function and execute the function.

RequireJS’s code loading capability actually relies on the dynamic generation and insertion of script tags, but it turns the process from manual to automatic. (You can read the RequireJS principle analysis if you are interested.)

The introduction of RequireJS addresses two pain points that previously existed when manually externalizing JS files:

  1. Manually maintain code reference order. RequireJS requires the user to specify the dependencies of each moduledependeciesArray auto-detection establishes dependencies between modules, ultimately helping users automate the insertion of Script tags on demand.
  2. Global variable pollution. RequireJS hides a large number of global variables in module files by encapsulating them in functions and providing only the return value (in effect, closures).

However, the implementation timing of THE AMD specification (factory executes all dependent modules first) and the dependency predeclaration (dependencies must be declared first, breaking the principle of nearest declaration) have aroused debate in the community. After several proposals to RequireJS were not adopted, Yubo himself wrote sea.js and the CMD (Common Module Definition) specification it follows. (Yubo related articles front-end modular development that point of history)

define(factory);

define(function (require.exports.module) {
  var$=require("jquery");
  $(document).ready(function(){$('#root') [0].innerText = 'Hello World'
  })
  exports.rootText = 'Hello World'
});
Copy the code

In the CMD specification, a module is a file. Unlike the AMD specification, CMD modules do not need to be declared in advance and only need to be executed when required. However, its basic implementation principle is similar to AMD and will not be repeated here. AMD/CMD is now buried in the dust of history.

From internal and external JS to the appearance of AMD/CMD modular front-end code, the front-end began to step out of the slash-and-burn era. But so far, all of these projects are essentially running directly in the browser, front-end engineering is still in its infancy, and a wide variety of front-end automation build tools are not yet available. (Note: Build tools do not include JAVA programs or Shell scripts that were used to compress code, merge files, etc.)

The rise of 1-NodeJS

In fact, long before the development of front-end automation build tools, front-end projects have built requirements, such as code compression, part of the file or image merge, but this function has to be done manually or through other languages/scripts.

Back in 2009, Ryan Dahl developed node.js as a JavaScript runtime based on the high performance and platform-independent nature of Google Chrome’s V8 engine. Since then, JS finally broke free from the bondage of the browser, began to have the ability to operate files. Node.js not only makes JS stand out in the server field and has a place, but more importantly, it really brings the front-end into the road of modern engineering.

Shortly thereafter, the first Node.js-based build tools began to emerge, with Grunt, the automated build tool, being the most widely used.

Grunt

Grunt automates repetitive tasks such as code compression, compilation, unit testing, linting, etc.

// Gruntfile.js
module.exports = function(grunt) {
  // Function configuration
  grunt.initConfig({
    // lint checks configuration
    jshint: {
      files: ['Gruntfile.js'.'src/**/*.js'.'test/**/*.js'].options: {
        globals: {
          jQuery: true}}},// Detect file changes and automatically execute related tasks
    watch: {
      files: ['<%= jshint.files %>'].tasks: ['jshint']}});// Load relevant task plug-ins
  grunt.loadNpmTasks('grunt-contrib-jshint');
  grunt.loadNpmTasks('grunt-contrib-watch');
  
	// List of tasks executed by default
  grunt.registerTask('default'['jshint']);
};
Copy the code

As you can see, Grunt is configuration-driven. All you need to do is learn what the various plugins do and integrate the configuration into gruntfile.js. However, this approach can increase configuration complexity dramatically when there are many tasks, and the code can look messy. Grunt’s I/O operation is also a disadvantage. It is slow to write files to disk at the end of each Grunt task and to read files from disk at the beginning of the next task.

Gulp

In order to solve the problems in using Grunt, Gulp, an automatic construction tool based on streaming, comes into being. The main feature of Gulp is that it introduces the concept of streams and provides a series of commonly used plug-ins to handle streams, which can be passed between plug-ins. This makes it simple on its own, but powerful enough to build on its own or in conjunction with other tools.

// gulpfile.js
const { src, dest } = require('gulp');
const babel = require('gulp-babel');

exports.default = function() {
  // Put all js files in the SRC folder into the output folder after Babel conversion
  return src('src/*.js')
    .pipe(babel())
    .pipe(dest('output/'));
}
Copy the code

With the rise of Node.js and the introduction of automated build tools like Grunt/Gulp, front-end engineering is getting back on track.

CommonJS – Synchronizes module loading

With the rise of Node.js, the CommonJS modular specification it follows has been accepted by most developers and has become the mainstream at that time. The require syntax used by CommonJS is synchronous. When a developer loads a module using require, it must wait for the module to load before executing the following code. The synchronous loading feature in node.js server only needs to read files from the local hard disk, which is relatively fast. However, in the browser side, due to the network delay, such loading mode is easy to make the page into a state of no response. Therefore, trying to use CommonJS in a browser is not going to work.

Browserify

Do developers need to use different modularity specifications in a single project? Not necessary! Where there is a need, there is a market, and Browserify is designed to help developers use CommonJS in the browser.

var browserify = require('browserify')
var b = browserify()
var fs = require('fs')

// Add the entry file
b.add('./browser/main.js')
// Package all modules into a file and export the bundle
b.bundle().pipe(fs.createWriteStream('./bundle.js'))
Copy the code

Browserify analyzes the AST at runtime to get the dependencies that exist between each module, generating a dependency dictionary. Each module is then wrapped, passing in a dependency dictionary and its own implementation of export and require functions, resulting in a JS file that can be executed in a browser environment. This process is often referred to as packaging.

Packing itself is a vague metaphor for what we pack after a meal at a restaurant. Sure, we can put all our leftovers in one plastic container to take home, but most of the time, we’ll put dishes that might be different into different plastic containers depending on the type of dish.

Generally speaking, we combine modular code from multiple files into one file and call it packaging. But due to the will of the whole project code into a file browser provided to perform the way will lead to web pages load too slowly for single file is too large, so the code division, there has been an increase in dynamic load demand began, finally formed the modular code together merge multiple files in the packaging in multiple files.

But because Browserify has a single responsibility for js module consolidation and packaging, as well as a code style similar to pipe functions and good gulp compatibility, developers often use them in combination. The Gulp+ Browserify build pattern was almost the accepted engineering standard on the front end for a while.

var browserify = require('browserify');
var gulp = require('gulp');
// Vinyl-source-stream can be used as a binder for both
// Use it to convert browserify generated files into gulp-supported streams
var source = require('vinyl-source-stream');
 
gulp.task('browserify'.function() {
    return browserify('./src/javascript/app.js')
        .bundle()
        // Pass desired output filename to vinyl-source-stream
        .pipe(source('bundle.js'))
        // Start piping stream to tasks!
        .pipe(gulp.dest('./build/'));
});
Copy the code

Sixth, ESM – the emergence of specifications

After years of fragmentation between AMD/CMD/CommonJS modularity specifications, the official JavaScript modularity standard ESM (ECMAScript Module) finally arrived in 2015. Unlike previous specifications, the ESM specification itself only explains how a file should be parsed into a module record, how to instantiate and evaluate the module, but does not require how to obtain the file, so the ESM supports both synchronous and asynchronous module loading methods. (Recommended reading ES Modules: A Cartoon Deep-dive, this is probably the most accessible, detailed, and accurate ESM presentation so far)

Webpack

Webpack, by far the most popular packaged build tool, must be mentioned here. The concept of Webpack was more engineering, and it didn’t catch on right away when it was launched, because front-end development wasn’t too complicated at that time, there were some MVC frameworks, but they were short-lived. The front-end stack is a choice between requireJs/ sea-js, Grunt /gulp, Browserify, and Webpack.

With the advent and rise of MVC frameworks and ESM, WebPack2 announced support for AMD CommonJS ESM, CSS/Less/Sass/Stylus, Babel, TypeScript, JSX, Angular 2 components and VUE components. There has never been such a large and comprehensive tool that supports so much functionality and can solve almost all of today’s build-related problems. The complexity of SPA applications makes WebPack the best choice with React/Vue/Angular, and webPack has truly become the heart of front-end engineering.

Webpack can be divided into three main parts:

  1. Main process: start the build, read the parameters, load the plug-in, then search for the dependent module from the entry file and compile it recursively, and finally package and output the build result.
  2. Loader: Processes and transforms the code in the files it matches according to the rules.
  3. Plugin: WebPack relies on Tapable for event distribution, registering hooks in plug-ins first, and then firing different hooks at different stages of the main process to execute different plug-ins.

Webpack is configuration based, and a simple configuration file example is shown below.

// webpack.config.js
const HtmlWebpackPlugin = require('html-webpack-plugin')

module.exports = {
    // SPA entry file
    entry: 'src/js/index.js'.output: {
      filename: 'bundle.js'
    }
    // Module matching and processing are mostly compilation processing
    module: {
        rules: [
  					// Babel transforms the syntax
            { test: /\.js$/, use: 'babel-loader' },
            / /...]},plugins: [
      	// Create an HTML file from the template
        new HtmlWebpackPlugin({ template: './src/index.html' }),
      	/ /...]./ /...
}
Copy the code

However, this large, all-in-one configuration model has the obvious downside of cumbersome configuration, and it quickly becomes annoying to some small project developers. For this group of developers, they are more likely to choose another small but elegant Rollup as their packaging tool.

Rollup

Rollup is packaged entirely based on the ESM module specification (and is also supported by plug-ins for CommonJS) and pioneered the concept of tree-shaking (later followed by WebPack). Simply put, ESM module dependencies are determined, independent of runtime state, and reliable static analysis is possible. Therefore, the packaging tool has the opportunity to separate out useless code at compile time and remove it), coupled with its simple configuration, easy to use, has become the most popular JS library packaging tool.

import resolve from 'rollup-plugin-node-resolve';
import babel from 'rollup-plugin-babel';

export default {
  // Import file
  input: 'src/main.js'.output: {
    file: 'bundle.js'.// Output module specifications
    format: 'esm'
  },
  plugins: [
    // Convert commonJS module to ESM
    resolve(),
    // Babel transforms the syntax
    babel({
      exclude: 'node_modules/**'}})]Copy the code

Rollup focuses on pure javascript and has less support for packaging, hot updates, and other features of non-JS resources than WebPack, so it is not used much in front end projects.

Seven, outside 2 – speed up construction

With the development of front-end engineering, the packaging time of more and more huge front-end projects is increasing. The packaging time of these projects is often several minutes or even more than ten minutes, which makes the performance of packaging tools more and more people pay attention to. Although V8 engine provides powerful performance support for JS operation, it is still unable to get rid of the performance shackles of JS interpreted language. In order to further improve the build performance, in recent years, there are some packaged build tools that do not use JS at the bottom, among which Esbuild is the most famous.

Esbuild

Esbuild base uses go language and makes extensive use of its high concurrency features, in terms of speed can be said to beat all JS packaging build tools on the market at present.

Esbuild supports the ES6/CommonJS specification, Tree Shaking, TypeScript, JSX, and more. Provides JS API/Go API/CLI call methods.

// JS API call
require('esbuild').build({
  entryPoints: ['app.jsx'].bundle: true.outfile: 'out.js',
}).catch(() = > process.exit(1))
Copy the code

Esbuild is still young, not 1.0, and its packaged build, like Rollup, is more JS focused, so it’s not suitable for use in a production environment on its own for front-end projects, but that doesn’t stop it from having a lot of potential.

ESM – Browser support

As the ESM specification matures and is now supported by major browsers, ESM modularity has become a built-in feature of browsers, so JS code does not actually need to be packaged. The following HTML file using the browser support ESM features, you can see at this stage the import import module can be run directly without any translation packaging in the new version of the chrome/firefox/edge/safari browser.

<! DOCTYPEhtml>
<head>
  <title>ESModule</title>
</head>
<body>
  <div id="root"/>
  <script type="importmap">
    {
      "imports": {
        "react": "https://cdn.pika.dev/react"."react-dom": "https://cdn.pika.dev/react-dom"}}</script>
  <script type="module">
    Bare import is supported
    import React from 'react'
    import ReactDOM from 'react-dom'
  
    ReactDOM.render('Hello! '.document.getElementById('root'))
  </script>
</body>
Copy the code

This has led to the emergence of development build tools such as Vite and Snowpack featuring the concept of budless.

Vite

Vite is the next generation front-end development and build tool developed by EVAN YOU. Its build process uses different approaches in development mode and production mode.

In development mode

When Vite starts development mode, it prebuilds it using Esbuild based on dependencies in the project, mainly to:

  1. Convert CommonJS modules to ESM modules
  2. Package each dependency into a single file to prevent a library like Lodash-ES from generating waterfall requests (lots of instantaneous concurrent file GET requests) when ESM was introduced

These dependencies change little and are cached in the. Vite directory after pre-build.

After that, Vite starts the development server without doing anything to the project code, and opens the project using the browser’s ESM support.

As you can see, in development mode, Vite is able to achieve this speed mainly due to the following factors:

  1. Esbuild is used for files that need to be packaged to build or compile transformations. Esbuild is written based on the GO language, and its performance is a hundred times that of traditional JS packaging tools.
  2. Dependency prebuild allows dependencies that take up most of the code size to be cached and rebuilt only at initial startup and dependency changes, saving dependency build time.
  3. Output ESM directly to the browser, eliminating the code packaging time of traditional development build tools before DEV Server starts.

In production mode

  1. Considering browser compatibility and the fact that using ESM on a real network can cause RTT to take too long, you still need to package the build.
  2. Because Esbuild is fast, but not stable at 1.0, and has weak support for code splitting and CSS handling, Rollup is still used for actual package builds in production environments, but it is likely to be changed to Esbuild in the future.

Rather than being a packaged build tool like Webpack, Think of Vite as a native ESM development server that takes care of converting unparsed files in browser ESM mode to ESM via third-party libraries, such as CommonJS to ESM and CSS to ESM. Most of its true file conversion and package build capabilities come through Esbuild or Rollup.

As you can see, while Vite has the potential for the next generation of development build tools, the current packaging process for development and production environments may lead to inconsistent project performance in the development environment and online.

Snowpack

SnowPack is a lightweight and fast front-end build tool developed by the Pika team. The team aims to make Web applications 90 percent faster. SnowPack is very similar to Vite in most of its features and ideas (both learn from each other, for example, Vite learned SnowPack from pre-build and SnowPack’s HMR learned from Vite). However, Snowpack is more idealistic and technology-oriented than Vite’s pragmatism.

There are two main differences between the two:

  1. SnowPack supports Streaming Imports.

Streaming Imports works very simply by converting local NPM package dependencies to NPM package dependencies in remote CDN.

// Original file
import "react"
 
// After compile (Streaming Imports)
import "https://pkg.snowpack.dev/react"
Copy the code

The benefits of this are as follows:

  • Because the remote NPM package has been packaged and compiled, the development build tool does not have to deal with dependencies, saving build time
  • Developers can run the dependency project without having to download and install it locally
  • The remote NPM package is distributed in the CDN edge node. When the user page is opened, the dependency will be downloaded nearby, saving the project loading time

SnowPack was able to think of and use this pattern because the Pika team had previously developed a project called SkyPack, which packaged NPM packages, compiled, compressed and uploaded to the CDN for developers to use.

  1. SnowPack uses Esbuild by default in production mode and exports ESM directly without packaging, consistent with development mode behavior. It does, however, allow users to choose from other packaged build tools such as Rollup and Webpack. (Note: More options doesn’t always mean better, Snowpack has more options but each mode has some problems, while Vite’s deep integration with rollup causes fewer problems.)

Nine,

Along the front modular change in a time line, this paper tells the story of the front-end build tools in the role of the each stage and its role, designed to allow the reader to seize its development in the wide variety of build tool change main vein, a rough understanding to its development trend, thus to make better choices.

Finally, we make a simple summary of the relevant technical tools in the article.

RequireJS Sea.js Grunt Gulp Browserify
Runtime modularity AMD CMD ❌ ❌ ❌
Compile-time modularity ❌ ❌ ❌ ❌ ✅ CommonJS
Automated build ❌ ❌ Configuration type File stream ❌
Project engineering support ❌ ❌ general general Generally, it is combined with Gulp
Webpack Rollup Esbuild Vite Snowpack
Runtime modularity ❌ ❌ ❌ ✅ ESM ✅ ESM
Compile-time modularity ✅ Supports all specifications ✅ ESM ✅ ESM ❌ ❌
Automated build ✅ ✅ ✅ ✅ ✅
Project engineering support ✅ optimal Generally, no HMR Generally, not used alone ✅ good ✅ good

In addition, the article only mentioned a few representative tools, in the development process of front-end build, there are many other tools or tools based on the secondary encapsulation of these tools, but everything is the same, as long as we master the rules can quickly distinguish the advantages and disadvantages of the building tool.