1. The background

As we all know, the size of the main package of small programs is limited to 2 meters, and developers are doing everything they can to optimize for increasingly large projects. The native small program basically uses GULP to build, whether to make an issue in the gulP construction process, to achieve the purpose of “optimization”.

Firstly, it is clear that the purpose of gulP construction optimization is to reduce the size of small program packages and improve user development experience. After getting familiar with the development of GULP, the author overthrew the original construction process, redesigned it, and optimized it for the purpose of improving, and made the following three progress:

1. Small program package volume reduction: the optimized pre-package volume 2212KB, the main package volume 1668.9KB; The optimized package volume is 2012KB and the main package volume is 1470.7KB. The package volume decreased by 9.04% and the main package volume decreased by 11.9%.
2. Shortened build time: dev mode build time is 58 seconds, build mode build time is 62 seconds before optimization; After optimization, the build time of dev mode and Build mode is 27 seconds and 32 seconds respectively. The build time of dev mode was reduced by 53.4% and that of Build mode was reduced by 48.4%.
3. Improved user development experienceTo optimize, click on the menu bar in the developer tools:tool –> Build the NPM, generate small program dedicatednpmThe package will compile the code successfully, and NPM will often have to be re-edited or rebuilt due to various path introduction problems. After optimization, there are no such worries, gulp build time automatically generated analysis applet dedicatednpmPackage, greatly improve the development efficiency.

2. Gulp workflow architecture

For small programs, in addition to app.js as a program entry, each page page can be used as a page entry, more inclined to be fixed path mode of multi-page applications. The purpose of gulp construction is to transfer the code translation from the development path to the special path for small programs. The code in this path can be read, compiled and constructed by wechat developer tools. Using the gulp tool:

• will.tsThe file is compiled to.js 、 .lessThe file is compiled to.wxssTo supportTypeScript 、 LessSyntax.
• supportsourcemapsConvenient error debugging and location.
• Compress pictures and all kinds of files, reduce the size of small program code package.
• Analyze code, rely on automatic extraction, support extraction commonnpmPackages and applets onlynpmThe package.
• Other files will be copied directly to the destination path.
• addwatch, convenient developer debugging.

2.1 split task

Js,. Json,. WXML, and. WXSS files. Ts and. Less files are usually introduced to improve the development efficiency. Because the compilation and construction methods of each file are different, different tasks need to be created for different types of files.

const src = './src';

// Files match paths
const globs = {
  ts: [`${src}/**/*.ts`.'./typings/index.d.ts'].// Matches the ts file
  js: `${src}/**/*.js`.// Matches the js file
  json: `${src}/**/*.json`.// Matches a JSON file
  less: `${src}/**/*.less`.// Matches less files
  wxss: `${src}/**/*.wxss`.// Matches the WXSS file
  image: `${src}/**/*.{png,jpg,jpeg,gif,svg}`.// Matches the image file
  wxml: `${src}/**/*.wxml`.// Matches the WXML file
  other: [`${src}/ * * `.`!${globs.ts[0]}`. ]// All other files except the above
};

// Create different tasks
const ts = cb= > {}; // Compile the ts file
const js = cb= > {}; // Compile the js file.const copy = cb= > {}; // Copy all other files to the target folder
Copy the code

2.2 Differentiation between Dev and Build modes

As a convention, we usually develop and debug in dev mode and release in build mode. These two gulp build schemes need to be distinguished. In dev mode, you need to add watch to listen for file changes and rebuild in time, and you need to add Sourcemap for debugging purposes. In build mode, you need to compress files to reduce package size.

// The default dev mode is configured
let config = {
  sourcemap: true.// Whether to enable sourcemap
  compress: false.// Whether to compress WXML, JSON, less and other files. };// Change the configuration to Build mode
const setBuildConfig = cb= >{ config = {... }; cb(); };// Execute all file build tasks concurrently
const _build = gulp.parallel(copy, ts, js, json, less, wxss, image, wxml);
// Build mode
const build = gulp.series(
  setBuildConfig, // Set the configuration to build mode
  gulp.parallel(clear, clearCache), // Clear the target directory files and cache
  _build, // Execute all file build tasks concurrently...). ;// build in dev mode
constBuild = gulp.series(clear,// Clear the target directory files
  _build, // Execute all file build tasks concurrently. watch,// Add a listener
);
Copy the code

3. The way to optimize

The previous section described the overall architecture of gulpfile. The following sections describe the specific configuration of each task and how to optimize it.

3.1 NPM construction optimization

3.1.1 Deficiencies of the official general scheme

① Install the NPM package

You can install the NPM package in either of the following ways:

1. manual: in the appletspackage.jsonRun commands in the directory where the command is locatednpm installInstall the NPM package, where those involved in building NPM are requiredpackage.jsonNeed to be inproject.config.jsThe definition of theminiprogramRootWithin.
2. Automatic transmissionThrough:gulpthetaskProcess and create onetask, will root directorypackage.jsonCopy the file to the directory where the applet resides (named in this article asminiprogram),execperformcd miniprogram && npm install --no-package-lock --productionCommand to install the NPM package. The code looks like this:

gulp.task('module:install'.(cb) = > {
	const destPath = './miniprogram/package.json';
	const sourcePath = './package.json';
	try {
		/ /... Omit the code to determine if package.json has changed, and return if it has not
		// Copy the file
		fs.copyFileSync(path.resolve(sourcePath), path.resolve(destPath));
		// Execute the command
		exec(
			'cd miniprogram && npm install --no-package-lock --production'.(err) = > {
				if (err) process.exit(1); cb(); }); }catch (error) {
		// ...}});Copy the code

② Build the NPM package

It is well known that we usenpm installBuild NPM packages will be therenode_modulesDirectory, but the applet specifiesnode_modulesThe directory does not participate in compilation, uploading, and packaging, so small programs must go through it to use the NPM package“Build NPM”Click on the menu bar in the developer tools:tool –> Build the NPM. At this time,node_modulesOne will be generated in the sibling directory ofminiprogram_npmDirectory, which will hold the build packed NPM package, which is the actual NPM package used by the applet.

As shown in the figure above, the structure of NPM packages used by applets is different from that of NPM packages in node_modules. This difference is the packaging process performed by clicking build NPM, which is divided into two types: The miniprogram NPM package directly copies all files in the build file generation directory to miniprogram_npm. Other NPM packages go through the dependency analysis and packaging process (similar to WebPack), starting with the entry JS file.

Obviously, the official NPM build solution exposes the following two problems:

1. Time consumingThrough:gulpWhen you package build applets, the first onetaskIs in theminiprogramIf the number of dependent NPM packages is large enough, it will consume a lot of time.
2. Cumbersome development processWhen debugging code using applets developer tools, re-click on the menu bar whenever a new NPM package is introduced or an old NPM package is updated:tool –> Build the NPM“Is very unfriendly to developers.

Therefore, we hope to use gulP workflow to analyze the dependencies of each file during the construction of the small program and copy the required NPM package to the miniprogram_npm directory. In one step, the above two steps of installing and building the NPM package are omitted.

3.1.2 Implementation of gulp-MP-NPM dependency analysis and extraction

Fortunately, the community can do anything, some authors developed a gulp plug-in gulp-mp-npm for small program extraction of NPM dependency package, has the following features:

• Dependency analysis only extracts the dependencies and components used.
• Support extraction of ordinary NPM packages and small program specific NPM packages.
• Dependencies will not be compiled or packaged (handed over to wechat developer tools or other gulp plug-ins).
• Compatible with official solutions and principles, and support custom NPM output folder.

As mentioned earlier, when building the NPM package, the NPM package will directly copy all the files in the build file generation directory into miniprogram_npm. For other normal NPM packages, the dependency analysis and packaging process (similar to Webpack) will be started from the entry JS file, and the generated code will generate source map file in the same directory for easy reverse debugging. However, the gulp-mp-npm plug-in only extracts the used dependencies and components through dependency analysis and copies them to the corresponding NPM package folder under miniprogram_npm directory, without compiling and packaging the dependencies. This step is left to wechat developer tools.

The construction principle of gulp-MP-NPM is shown in the figure above. For further understanding, you can refer to: Gulp plug-in design for small program NPM dependency package.

The use of plug-ins can be configured in gulpfile.js according to project requirements:

const gulp = require('gulp');
const mpNpm = require('gulp-mp-npm')

const js = () = > gulp.src('src/**/*.js')
    .pipe(mpNpm()) // Analyze the dependencies used in extracting JS
    .pipe(gulp.dest('dist'));

const less = () = > gulp.src('src/**/*.less')
    .pipe(gulpLess()) / / compile less
    .pipe(rename({ extname: '.wxss' }))
    .pipe(mpNpm()) // Analyze the dependencies used to extract less
    .pipe(gulp.dest('dist')); .Copy the code

NPM dependencies can be used in.ts,.js,.json,.less,.wxss, etc. Therefore, the plug-in gulp-MP-npm is executed in all five tasks. The.json file is analyzed because the plug-in will try to read the usingComponents field in the applet page configuration to extract the NPM applet component used.

3.1.3 Data comparison

3.2 Watch incremental compilation

It is well known that gulp tasks are executed once. After a file changes, gulp.watch will re-execute the entire task to complete the build, which results in repeated builds of unchanged files and low performance. The following two measures can be taken to optimize efficiency:

3.2.1 splittask, reasonable creationwatch

As mentioned above, according to different file types, eight tasks are divided into copy, TS, JS, JSON, less, WXSS, image and WXML, and watch is created for each task. If the index.ts file is modified, only the TS task is executed again, and other tasks are not affected. The code looks like this:

const watchOptions = { events: ['add'.'change'.`unlink`]};const watch = () = > {
  gulp.watch(globs.copy, watchOptions, copy);
  gulp.watch(globs.ts, watchOptions, ts);
  gulp.watch(globs.js, watchOptions, js);
  gulp.watch(globs.json, watchOptions, json);
  gulp.watch(globs.less, watchOptions, less);
  gulp.watch(globs.wxss, watchOptions, wxss);
  gulp.watch(globs.image, watchOptions, image);
  gulp.watch(globs.wxml, watchOptions, wxml);
};
Copy the code

3.2.2 gulp.lastRunImplementing incremental compilation

Incremental compilation is an essential optimization in any build tool. That is, compiling only those files that have been modified during compilation reduces unnecessary resource consumption and compilation time. Before the release of Gulp 4, the community came up with a series of solutions: gulp-changed, gulp-cached, gulp-remembered, gulp-newer, etc. Gulp 4 comes with an incremental update to gulp.lastrun ().

The gulp.lastrun method returns the last time the task successfully completed in the current running process. Pass this in as the parameter since to the gulp.src method, and compare the mtime(the last time the file contents were modified) of each file with the value passed in since to skip files that have not changed since the last successful completion of the task, achieving incremental compilation and speeding up execution time. The usage method is as follows:

/* For ts/less, js/json/WXSS/copy/image */
const ts = () = > gulp.src(
    'src/**/*.ts',
    { since: gulp.lastRun(ts) }
)...

const less = () = > gulp.src(
    'src/**/*.less',
    { since: gulp.lastRun(less) }
)...
Copy the code

However, this approach does incremental compilation only by identifying changes to the contents of the file. What about moving unmodified files? For example, when a JS file is copied to another directory, its Mtime (the time when the file contents were last modified) will not change. In this case, ctime(the time when the file was last modified to write to the file, change the owner, permissions, or link Settings) will come in handy. The code changes as follows, encapsulating the since function so that when the contents of the file are unchanged but the file path is changed, the timestamp is returned as 0, and the file can be incrementally compiled by comparing the mtime of the file (when the contents of the file were last modified) with the value passed in since (which is now 0).

const since = task= > file= >
  gulp.lastRun(task) > file.stat.ctime ? gulp.lastRun(task) : 0;

const ts = () = > gulp.src(
    'src/**/*.ts',
    { since: since(ts) }
)
Copy the code

3.2.3 Data comparison

3.3 open sourcemap

Gulp-sourcemaps is a plug-in used to generate a mapping file. The SourceMap file records an information file that stores the mapping between source code and compiled code. We can’t debug as easily as source code, so we need SourceMap to help us convert source code in the console for debugging. Gulp-sourcemaps are used to address code obturation and typescript and less language conversions to JS and CSS.

Use gulp-sourcemaps to enable Source Map for.ts,.less files:

const sourcemaps = require('gulp-sourcemaps');

/* For example, less */
const ts = () = > gulp.src('src/**/*.ts')
    .pipe(sourcemaps.init())
    .pipe(tsProject())  / / compile ts
    .pipe(mpNpm())      // Analyze the dependencies used to extract ts
    .pipe(sourcemaps.write('. '))   // Export it to the same directory as a.map file
    .pipe(gulp.dest('dist'));
Copy the code

Note: The Source Map file does not count into the code package size calculation, that is, the compiled and uploaded code does not count this part of the volume. Due to the inclusion of the.map file in the development code package, the actual code package size will be larger than the trial and official versions.

3.4 compile ts

The goal of compiling ts is to convert.ts files into.js files and output them to the target folder. It is mainly divided into the following steps:

1. Create a stream to read from the file systemVinylObject. Set up thesinceAttribute, usinggulp.lastRunIncremental compilation.
2. The introduction ofgulp-ts-aliasThe plug-in handles the path alias problem according to the plug-intsconfig.jsonIn the filepathsProperty to replace the alias with the original path. Such aspathsOne configuration is as follows"@/*": ["src/supermarket/*"], so for any.tsIn the fileimport A from '@/components'Will be replaced byimport A from 'src/supermarket/components'.
3. In dev mode, sourcemap is enabledgulp-ifTo make a conditional judgment whenconfig.sourcemapThis parameter is executed only when the value is truesourcemaps.init()Otherwise, the flow goes directly to the next pipe.
4. The introduction ofgulp-typescriptThe plugin will.tsFile conversion compiled into.jsFile. First, intsthistaskUse outsidegulpTs.createProjectCreate a TS compilation task. The reason why it is created outside is when runningwatchThere are.tsThe file is modified and needs to be rebuilttsthistask, creating it outside can save half the time. With the default configuration, a compilation error with TS is printed to the console, and the compiler crashes the build task because of the compilation error. So you need to be able totsProject()Add an error handler later to catch errors.
5. The introduction ofgulp-mp-npmPlug-in extract.tsFile imported NPM package.
6. willsourcemapWrite to the destination directory.
7. Import in build modegulp-uglifyright.jsThe file is compressed.
8. Will compile complete.jsThe file is output to the corresponding directory.

const gulpTs = require('gulp-typescript');
const tsAlias = require('gulp-ts-alias');
const gulpIf = require('gulp-if');
const uglifyjs = require('uglify-js');
const composer = require('gulp-uglify/composer');
const minify = composer(uglifyjs, console);
const pump = require('pump');

const tsProject = gulpTs.createProject(resolve('tsconfig.json'));   // 4. Create a ts compilation task externally

const ts = cb= > {
  consttsResult = gulp .src(globs.ts, { ... srcOptions,since: since(ts) }) // 1. Incremental compilation
    .pipe(tsAlias({ configuration: tsProject.config })) // 2. Replace the path alias with the original path
    .pipe(gulpIf(config.sourcemap, sourcemaps.init()))  // 3. Enable sourcemap in dev mode
    .pipe(tsProject())     // 4. Compile ts
    .on('error', （） => {    // 4. Catch the error, do not add the task will interrupt because of the TS compilation error
      /** Ignore compiler error **/
    });

  pump(
    [
      tsResult.js,
      mpNpm(mpNpmOptions),     // 5. Analyze dependencies
      gulpIf(config.sourcemap.ts, sourcemaps.write('. ')), // 6. Write the sourcemap file to the corresponding directory
      gulpIf(config.compress, minify({})),     // 7. Build mode compression js
      gulp.dest(dist),   // 8. Output files to the destination directory
    ],
    cb,
  );
};
Copy the code

Sharp-eyed readers will have noticed that the second half of task uses pump to link streams together. Pump is a small node module that connects streams together and destroys them all when any one of them shuts down. Pipe is used instead of pipe, and is ideal for fixing gulp-uglify errors. As shown in the following figure, a pump error can pinpoint the exact location, whereas pipe throws up the entire call stack, making it confusing.

3.5 compile less

The goal of compiling less is to convert.less files into.wxss files for output to the target folder. It is mainly divided into the following steps:

const gulpLess = require('gulp-less');
const weappAlias = require('gulp-wechat-weapp-src-alisa');
const prettyData = require('gulp-pretty-data');

const less = cb= > {
  pump(
    [
      gulp.src(globs.less, { ...srcOptions, since: since(less) }), // 1. Incremental compilation
      gulpIf(config.sourcemap.less, sourcemaps.init()), // 2. Enable sourcemap
      weappAlias(weappAliasConfig), // 3. Replace the path alias with the original path
      /** step A */ is omitted
      gulpLess(), // 4. Compile less into CSS
      /** omit step B */
      rename({ extname: '.wxss' }), Change the suffix of file. less to. WXSS
      mpNpm(mpNpmOptions), // 6. Dependency analysis
      gulpIf(config.sourcemap.less, sourcemaps.write('. ')), // 7. Write sourcemap
      gulpIf(
        config.compress,  Compress WXSS in build mode
        prettyData({
          type: 'minify'.extensions: {
            wxss: 'css',
          },
        }),
      ),
      gulp.dest(dist), // 9. Export files to the destination directory
    ],
    cb,
  );
};
Copy the code

As shown above, the gulp-less plugin is used to compile less code into CSS code, which is compiled as follows:

• index.lessThe file importvariable.lessVariable file, will bevariable.lessCopy the content toindex.lessFile.
• index.lessThe file importstyle.lessThe pure style file willstyle.lessCopy all the content toindex.lessFile.
• .lessFile compilation, will be used to replace the variable with the corresponding value; Style nested tiling.
• emptystyle.lessandvariable.lessThe contents of the file.

Imagine that 100 files are imported into the Style. less pure style file and 100 copies of the style.less content are made. For complex engineering projects, the dependency of less files can be complex, and the result of compilation is a lot of redundant style code, which is the opposite of our philosophy of “keep improving” (minimizing packages as much as possible). For this reason, we can comment out the code related to @import ** before gulp-less is compiled. After gulp-less is compiled, we can restore the content of the comment and change the suffix of the introduced path to.wxss. The code is as follows:

/** Step A omitted from the previous code snippet */
tap(file= > {
  const content = file.contents.toString(); // toString()
  const regNotes = /\/\*(\s|.) *? \*\//g;   // Match /* */ comment
  const removeComment = content.replace(regNotes, ' '); // Delete the comment content
  const reg = /@import\s+['|"](.+)['|"]; /g; // matches @import ** path import

  const str = removeComment.replace(reg, ($1, $2) = > {
    const hasFilter = cssFilterFiles.filter(item= > $2.indexOf(item) > -1);  // Filter out variable file import
    let path = hasFilter <= 0 ? `/** less: The ${$1}* * / ` : $1;  /** less: ${$1} **/
    return path;
  });
  file.contents = Buffer.from(str, 'utf8'); // String restores to a file stream
});
Copy the code

Note that if you comment out a variable file, such as the variable.less file mentioned above, then an imported variable such as @color-primary will not take its value, resulting in a compilation error. Therefore, you can write an array of cssFilterFiles to filter out the variable files, and then comment out all the style files, such as the style.less pure style file mentioned above.

After executing step A code, use gulp-less to compile the less code into CSS code. After executing step B code, as shown below, restore the path comments and change the. Less suffix of the introduced path to the. WXSS suffix that can be recognized by the applet.

/** Step B code omitted from the previous code snippet */
tap(file= > {
  const content = file.contents.toString();
  const regNotes = /\/\*\* less: @import\s+['|"](.+)['|"]; \*\*\//g;
  const reg = /@import\s+['|"](.+)['|"]; /g;
  const str = content.replace(regNotes, ($1, $2) = > {
    let less = ' ';
    $1.replace(reg, $3 => (less = $3));
    return less.replace(/\.less/g.'.wxss');
  });
  file.contents = Buffer.from(str, 'utf8');
});
Copy the code

The optimized effect is shown in the figure below:

3.6 Image Compression

The applets main package currently only supports a maximum size of 2M, and images are usually the most space-consuming resource. It is necessary to compress the image size in a project.

Using gulp-image, you can compress the image size and guarantee the image quality:

const gulpImage = require('gulp-image');
const cache = require('gulp-cache');

const image = () = >gulp .src(globs.image, { ... srcOptions,since: since(image) })
    .pipe(cache(gulpImage())) // Cache compressed images
    .pipe(gulp.dest(dist));
Copy the code

As shown in the figure below, it is the result of compression of all pictures of the project, which can save about 50% volume on average.

3.7 Compiling Other Files

.js,.json,.wxml, and.wxss are compiled by copying the source file to the target file directory. The code looks like this:

const prettyData = require('gulp-pretty-data');

const wxml = () = >gulp .src(globs.wxml, { ... srcOptions,since: since(wxml) }) // 1. Incremental compilation
    .pipe(
      gulpIf(
        config.compress,    // 2. Compress files in build mode
        prettyData({
          type: 'minify'.extensions: {
            wxml: 'xml',
          },
        }),
      ),
    )
    .pipe(gulp.dest(dist)); // 3. Output the output to the corresponding directory

//. Json,. WXML,. WXSS code refer to above
Copy the code

In wechat developer tools, click details in the upper right corner -> Local Settings -> check the confusion between automatic compression style when uploading code and automatic compression when uploading code, then. WXSS and.js files will be compressed when applet is built and packaged, so in the actual gulp build workflow, You can compress. Ts,. Js,. Less, and. WXSS, but for. WXML and. Json files, you can use the gulp-pretty-data plug-in to compress the files.

4. The last

The biggest limitation of the development of small programs is that the size of the main package must be controlled within 2M. Through the optimization method in this paper, the volume of the original main package of small programs is reduced by 11.9%. To further reduce the volume, tree-Sharking can be considered to analyze code dependencies and eliminate unreferenced codes. However, the existing gulP workflow architecture is difficult to implement this function perfectly. Tree-sharking is the strength of rollup, WebPack and other build solutions, which is why some mature apet frameworks such as Taro and MPvue choose WebPack to build. In fact, each has his or her own strengths.

Attached code links: codesandbox. IO/s/miniprogr…