Vite2 source analysis (two) – request resources

In this article, we mainly introduce the process of starting Vite, how the 衱 plugin mechanism works, how the module is stored and managed, and how the detailed pre-build process looks like. This article focuses on what the browser does when it starts requesting a module. This example project is also based on the project set up in the background of the first article.

Above is the flowchart of the entire request, and below we will analyze step by step what is done at each step.

Browser access to resources

When the loaded HTML has a script tag of type Module, SRC is the entry js file to the entire project:

<script type="module" src="/src/main.js"></script>
Copy the code

Browser to identify the script for the module type, so a network request http://localhost:3000/src/main.js file/SRC/main js. The request comes to the Connect service created by Vite, and similar to KOA, the request URL starts to be processed through layers of middleware, one of the more important and difficult middleware for JS is the middleware transformMiddleware.

Source location: vite v2.4.1 / packages/vite/SRC/node/server/middlewares/transform. The ts

Do some processing to the requested URL first, remove t, import such as query, and then remove __x00__ prefix, get a relatively pure URL.

Determine whether to cache

The next step is to decide whether to use caching directly, which is usually browser-strong for modules that access node_modules. Judge strong cache source code as follows:

  const ifNoneMatch = req.headers['if-none-match']
    if (
      ifNoneMatch &&
      (awaitmoduleGraph.getModuleByUrl(url))? .transformResult? .etag === ifNoneMatch ) { isDebug && debugCache(` [304]${prettifyUrl(url, root)}`)
      res.statusCode = 304
      return res.end()
    }
Copy the code

This module is obtained from the module diagram by URL. As explained in the first article, each module has an index to the urlToModuleMap object, so accessing the module is very efficient. Gets the module’s ETag, which is generated when the module is created and will be covered later. If the eTAG in the browser request header is the same as the eTAG in the module cache, then it is possible to reuse the cache to return the 304 status code directly, otherwise the subsequent module request process is required.

Request module – transformRequest

The module’s procedure is rerequested when it has previously determined that the cache is not available or not available.

Judge the cache again

 const module = await moduleGraph.getModuleByUrl(url)
  const cached =
    module && (ssr ? module.ssrTransformResult : module.transformResult)
  if (cached) {
    return cached
  }
Copy the code

Retrieve the module from the module cache via the URL. If the module is present, it means that the browser requested the module is old, but during file update or HMR the module has been rebuilt and cached again, so it can be returned directly.

Get the resolveId of the module from the URL

const id = (awaitpluginContainer.resolveId(url))? .id || urlCopy the code

Without the cache, you need to go through the process of creating the module and adding it to the module cache. First, obtain the resolveId of the module through the URL. This step is to execute the resolveId method of the plug-in container. This method is described in the previous article in order to execute the plug-in’s resolveId method, and then get the id first. Specifically, there are two plug-ins that execute this method:

Vite: pre – alias plug-in

Source location: vite/v2.4.1 / packages/vite/SRC/node/plugins/preAlias ts

const bareImportRE = /^[\w@](? ! . * : \ \ / /
export function tryOptimizedResolve(id: string, server: ViteDevServer) :string | undefined {
  const cacheDir = server.config.cacheDir
  / / metadata. Json file
  const depData = server._optimizeDepsMetadata
  if (cacheDir && depData) {
    const isOptimized = depData.optimized[id]
    if (isOptimized) {
      return (
        isOptimized.file +
        `? v=${depData.browserHash}${
          isOptimized.needsInterop ? `&es-interop` : ` `
        }`)}}}resolveId(id, _, __, ssr) {
// If it is a third-party module such as Vue lodash-es
    if(! ssr && bareImportRE.test(id)) {return tryOptimizedResolve(id, server)
    }
  }
  
Copy the code

TryOptimizedResolve (resolveId) ¶ tryOptimizedResolve (resolveId) ¶ tryOptimizedResolve (resolveId) ¶ The main method is to get the pre-built metadata.json file from the cache directory (.vite) and the file address of the third-party module. The third party module’s Vite resolveId is then formed by file, browerHash Query, and whether it is a CJS Query. So if you have in your code

import { createApp } from "vue";
Copy the code

The resolveId is /[root]/node_modules/.vite/vue.js.

Vite: resolve plug-in

/ /...
// Start with /, absolute path
if (asSrc && id.startsWith('/')) {
  const fsPath = path.resolve(root, id.slice(1))
  if ((res = tryFsResolve(fsPath, options))) {
    return res
  }
}
 / /...
 // Relative path starting with
 if (id.startsWith('. ')) {
 // The absolute path back to the module relative to the path of the imported file
  const basedir = importer ? path.dirname(importer) : process.cwd()
  const fsPath = path.resolve(basedir, id)

  const normalizedFsPath = normalizePath(fsPath)
  const pathFromBasedir = normalizedFsPath.slice(basedir.length)
  // If you get node_nodules relative to root, this means that node_modules files are accessed directly through a relative path
  if (pathFromBasedir.startsWith('/node_modules/')) {
    // normalize direct imports from node_modules to bare imports, so the
    // hashing logic is shared and we avoid duplicated modules #2503
    const bareImport = pathFromBasedir.slice('/node_modules/'.length)
    if (
      (res = tryNodeResolve(
        bareImport,
        importer,
        options,
        targetWeb,
        server,
        ssr
      )) &&
      res.id.startsWith(normalizedFsPath)
    ) {
      return res
    }
  }

  // non-node_modules go back to the absolute path
  if ((res = tryFsResolve(fsPath, options))) {
    isDebug && debug(`[relative] ${chalk.cyan(id)} -> ${chalk.dim(res)}`)
    constpkg = importer ! =null && idToPkgMap.get(importer)
    if (pkg) {
      idToPkgMap.set(res, pkg)
      return {
        id: res,
        moduleSideEffects: pkg.hasSideEffects(res)
      }
    }
    return res
  }
}
Copy the code

The plug-in handles requests for regular JS files, and for absolute path requests, returns the absolute path address rooted in the root directory. For a relative path request, obtain an absolute address relative to the address of the imported file, and obtain an absolute path rooted in the root directory.

For example, in the main file:

import App from "./App.jsx";
Copy the code

We get:

import App from "/src/App.jsx";
Copy the code

tryNodeResolve

const resolve = require('resolve'); function tryNodeResolve( id, importer, options, targetWeb, server, ssr ) { const { root, dedupe, isBuild } = options const deepMatch = id.match(/^([^@][^/]*)/|^(@[^/]+/[^/]+)//); const pkgId = deepMatch ? deepMatch[1] || deepMatch[2] : id; let basedir if (dedupe && dedupe.includes(pkgId)) { basedir = root } else if ( importer && path.isAbsolute(importer) && fs.existsSync(cleanUrl(importer)) ) { basedir = path.dirname(importer) } else { basedir = root } const pkg = resolvePackageData(pkgId, basedir) if (! pkg) { return } let resolved = deepMatch ? resolveDeepImport(id, pkg, options, targetWeb) : resolvePackageEntry(id, pkg, options, targetWeb) if (! resolved) { return } return { id: resolved }; }; export function resolvePackageData( id, basedir ) { try { // https://www.npmjs.com/package/resolve const pkgPath = resolve.sync(`${id}/package.json`, basedir) return loadPackageData(pkgPath) } catch (e) { } }; function loadPackageData(pkgPath) { const data = JSON.parse(fs.readFileSync(pkgPath, 'utf-8')) const pkgDir = path.dirname(pkgPath) const { sideEffects } = data let hasSideEffects if (typeof sideEffects === 'boolean') { hasSideEffects = () => sideEffects } else if (Array.isArray(sideEffects)) { hasSideEffects = createFilter(sideEffects, null, { resolve: pkgDir }) } else { hasSideEffects = () => true } const pkg = { dir: pkgDir, // [root]/node_modules/vue data, hasSideEffects, webResolvedImports: {}, nodeResolvedImports: {}, setResolvedCache(key: string, entry: string, targetWeb: boolean) { if (targetWeb) { pkg.webResolvedImports[key] = entry } else { pkg.nodeResolvedImports[key] = entry } }, getResolvedCache(key: string, targetWeb: boolean) { if (targetWeb) { return pkg.webResolvedImports[key] } else { return pkg.nodeResolvedImports[key] } } } return  pkg }Copy the code

When tryNodeResolve is called with id vue, resolvePackageData (resolvePackageData) is called with id vue. Resolve is called based on id and baseDir. Dir is [root]/node_modules/vue. Then call resolveDeepImport, which reads package.json from the vue module. For the main, the module entry file information, and then return to the correct path/root/node_modules/vue/dist/vue runtime. Esm – bundler. Js

Get the code by resolveId

const loadResult = await pluginContainer.load(id, ssr)
Copy the code

In the case of js files, this is the contents of the file read using the fs.readfile module and then returned. Load, like the resolveId module, is also the plug-in container’s execution hook. It executes each plug-in’s load method in sequence according to the order of the plug-in. If any content is returned, it will return directly.

Get the final code by code Transform

  const transformResult = await pluginContainer.transform(code, id, map, ssr)
Copy the code

In the transformRequest file, we pass in the code obtained from the load and the ID obtained from the resolveId to enter the transform phase of the plug-in container. At this stage, multiple plug-ins will be executed sequentially. The result of each plug-in is the input parameter of the next plug-in, and the final processed code will be returned after all plug-in processing. The most important plug-in we have here is the Vite: import-Analysis plug-in, which is designed to replace the path of importing modules in code. Details are as follows:

vite:import-analysis

Source location: vite/v2.4.1 / packages/vite/SRC/node/plugins/importAnalysis ts

The process is roughly as shown in the figure above. The source code will not be pasted in this part. The process is complicated and will be replaced by a flow chart:

We first parse the code using esbuild’s Parser module to get the module’s dependencies. If the module has no dependencies, it returns code directly, without further dependency processing, which would otherwise be required for each import.

For each import, import path to remove the base part, base usually is /, then through introducing before pluginContainer. ResolveId resolveId (url) to get to rely on the module, Then convert the ID to a URL that can be written to the file, if the ID starts with a/(because resoveId is an absolute path, you need to remove the root part of the path). Then moduleGraph. EnsureEntryFromUrl (url).

In ensureEntryFromUrl, if the module cache does not have a cache for this URL, create a module and get the resolveId,

File is added to urlToModuleMap, idToModuleMap, and fileToModulesMap. If the file exists, the object of this module is directly returned. It is convenient to rewrite code, etag and other information later.

Then rewrite the path reference from the rewritten import XXX from “XXX” form to the resulting URL and return the rewritten code.

Reset module, response content

 return (mod.transformResult = {
    code,
    map,
    etag: getEtag(code, { weak: true})}as TransformResult)
Copy the code

After executing the resoveId, Load, and Transform hooks above, we finally get the parsed code and map, and we also get the module cache various corresponding Module objects. We then reset the Module’s transformResult property. Reset the code, map, regenerate an ETag based on the code content, and then respond to the content.

export function send(
  req: IncomingMessage,
  res: ServerResponse,
  content: string | Buffer,
  type: string,
  etag = getEtag(content, { weak: true }),
  cacheControl = 'no-cache', map? : SourceMap |null
) :void {
  if (req.headers['if-none-match'] === etag) {
    res.statusCode = 304
    return res.end()
  }

  res.setHeader('Content-Type', alias[type] || type)
  res.setHeader('Cache-Control', cacheControl)
  res.setHeader('Etag', etag)

  // inject source map reference
  if (map && map.mappings) {
    if (isDebug) {
      content += `\n/*The ${JSON.stringify(map, null.2).replace(
        /\*\//g.'* \ \ /'
      )}*/\n`
    }
    content += genSourceMapString(map)
  }

  res.statusCode = 200
  return res.end(content)
}

function genSourceMapString(map: SourceMap | string | undefined) {
  if (typeofmap ! = ='string') {
    map = JSON.stringify(map)
  }
  return `\n//# sourceMappingURL=data:application/json; base64,${Buffer.from(
    map
  ).toString('base64')}`
}
Copy the code

The response is to set the content-type, eTag, For cache-control, max-age=31536000 for node_modules, which are scanned in pre-build,immutable is no-cache for common modules. Set the 200 response code to return the file content, but add the sourcemAP content at the end of the file content to improve performance. The file request processing process ends at this point.