This article takes 7 minutes to read. For more articles, please visit my bloghalu886

  • introduce
  • Train of thought
    • To prepare
    • Prepositional logic
      • chunkSetsInGraph
      • chunkSetsByCount
      • combinationsCache
      • selectedChunksCacheByChunksSet
    • The core
  • conclusion

introduce

Chunks (collections of modules) are related in parent-child relationships in dependency diagrams parsed by Webpack. The CommonsChunkPlugin was originally designed to avoid duplicate dependencies between chunks, but performance was far from optimal.

In WebPack 4, the ChunkSplitPlugin is built in to replace the CommonsChunkPlugin.

The following is a summary based on the official demo, combining the main data structures with diagrams and personal understanding.

Uncover the mystery of the design and source code of this plug-in

Train of thought

To prepare

It is recommended to take a look at the TAP-based event flow architecture within WebPack before combing through it. As well as the data structure of Chunk(a collection of modules, which is also the packaging unit of Webpack, and the final output file is Chunk) and Module(the parsing unit in the dependency tree, which can be simply understood as each dependent JS file)

Looking at the file structure in the demo, you can easily draw the following conclusions

  • There are 7 entry files a ~ F. js, i.e., at least 7 chunks (represented by indexes 1 ~ 7 respectively)
  • There are 14 JS files, so there are 14 modules
  • There is no interdependence between s1.js~s8.js and node_modules/m1~m7.js on stuff/*

The plugin is injected into the Compiler’s thisCompilation TAP event and compilation’s optimizeModules TAP event

thisCompilation:Executed while initializing the compilation, right before emitting the compilation event. optimizeModules:Called at the beginning of the module optimization phase. A plugin can tap into this hook to perform optimizations on modules.

In order to facilitate subsequent debugging and combing, the dependency relationship between 14 files of demo is specially sorted out

The seven entryPoints are separated by color. The arrows point to dependencies and the orange area is newChunk generated by ChunkSplitPlugin

And all Chunk snapshots before the execution

./webpack.config.js

module.exports = {
  // mode: "development || "production",
  entry: {
    pageA: "./pages/a".pageB: "./pages/b".pageC: "./pages/c".pageD: "./pages/d".pageE: "./pages/e".pageF: "./pages/f".pageG: "./pages/g",},optimization: {
    splitChunks: {
      chunks: "all".maxInitialRequests: 20.// for HTTP2
      maxAsyncRequests: 20.// for HTTP2
      minSize: 40.// for example only: chosen to match 2 modules
      // omit minSize in real use case to use the default of 30kb,}}};Copy the code

Prepositional logic

Before the plug-in analyzes repeated dependencies between the entire dependency graph collating modules, a lot of additional optimizations are made to cache most of the data state by swapping space for time.

The following is a step-by-step breakdown through code order and logic.

chunkSetsInGraph

Through traversing 14 modules, analyze the Chunk that module.chunksIterable is dependent on, integrate the key value pointing to the collection of Chunk through the index of Chunk, the type is Map

>
,>

For example, when traversing m1, the module is referenced by Chunk1 and Chunk2 as well as Chunk3, as shown in figure “1,2,4”->set

{Chunk1, Chunk2, Chunk3}.

When traversing m2, it is found that the modified module is also referenced by Chunk1 and Chunk2 as well as Chunk3, and “1,2,3” has been cached, so the traversal will be skipped and continue

chunkSetsByCount

Iterate over the chunkSet Singraph, aggregating the chunkSet through length. Map

>>
,>

For example, when traversing ChunkSetsInGrouph, the length of Set

{Chunk1,Chunk3} pointed to by “1,3” is 2. The chunkSetsInGraph is pushed with a key of 2 pointing to the [{Chunk1,Chunk3}] array. Length 2 when traversing “2,5” to Set

{Chunk2,Chunk5}, [{Chunk1,Chunk3},{Chunk2,Chunk5}]; [Chunk2,Chunk5}];

combinationsCache

All chunksets are traversed, aggregating the ChunkSet and its subsets, and referencing the ChunkSet maximum key in the aggregation. Map

[]>
,>

Such as: Traverse chunkSetInGraph, when key value is 1,2,3, point to [{chunk1,chunk2,chunk3}], and push all subSet into subsequent traverse, Finally, “1,2,3” points to [{chunk1,chunk2,chunk3},{chunk1},{chunk2},{chunk3}]

selectedChunksCacheByChunksSet

All chunksets correspond to different configurations filtered based on webpack.config.js. Type WeakMap

, WeakMap

>
,>

In our demo “chunks: “all” configuration, all filterfunctions are const ALL_CHUNK_FILTER = chunk => true; , so SelectChunkResult is all chunkSet itself

The core

Based on the cached data generated above, the next step is to start generating the most important data structure, chunksInfoMap, while traversing this object to peel off new chunks and establish connections in the ChunkGroup. Type Map < string, ChunksInfoItem >, ChunksInfoItem type {modules, cacheGroup, name, chunks, chunksKeys, blabla… }

For modules, combinationCache and cacheGroups are three layers of nested loop find repeat each is dependent on the module of the chunk relationship (can try to control the first figure reverse deduction)

CacheGroups is an intermediate configuration item generated when each module is traversed. It is used to distinguish module types and filter modules. For example, in the chunk:all configuration, modules are configured {key:normal,minChunk: 2}, but m1 to m7 under node_modules/* additional {key:vendor,minChunk: 1}

Such as: Traversing through M1, there are two cacheGroup configuration items [{key:normal},{key:vendors}], And is dependent in combinationCache by {chunk1,chunk2,chunk3} and subset [{chunk1,chunk3},{chunk1},{chunk2},{chunk3}]. Therefore, 7 pieces of data are respectively pushed based on key value (the other 3 pieces are filtered because vendor’s minChuck is 2).

Finally, the chunksInfoMap is looping

At each loop retrieval, the ChunksInfoItem with the highest priority is pushed (by name/path/chunks.lengthd length, etc.).

After checking, using the compilation. AddChunk generates a newChunk, and GraphHelpers. ConnectChunkAndModule newChunk injection in the Graph, In addition, the module in newChunk is removed using chunk.removeModule.

Until chunksInfoMap is empty

Note: Before entering the next loop, the remaining ChunksInfoItem modules will be cleaned up and the modules that have been encapsulated into newChunk will be removed. If the Module of ChunksInfoItem is 0, The chunkInfoItem is deleted

conclusion

Webpack greatly liberates the front-end engineering labor force, a plug-in logic can give us a lot of inspiration, and for reading well-known open source projects, the source code structure is indeed very rigorous and beautiful, reap a lot.

Because the demo is relatively basic, many boundary cases in the plug-in are not involved, but the main process combing is relatively perfect.

Studying master code is like reading a book. It’s painful but enjoyable