This article was first published in the public number ByteDance Web Infra pay attention to, the follow-up on the new good ~, team introduction: webinfra.org/about

Web. dev/bundling-no…

Suppose you are developing a web application. In this case, you’re likely to be dealing not only with JavaScript modules, but also with a variety of other resources — Web Workers (which is also JavaScript, but has a separate set of build dependency diagrams), images, CSS, fonts, WebAssembly modules, and so on.

One possible way to load static resources is to reference them directly in HTML, but often they are logically coupled to other reusable components. For example, the CSS for the custom drop-down menu is associated with its JavaScript section, the icon image is associated with the toolbar component, and the WebAssembly module is dependent on its JavaScript glue. In these cases, it is much easier to reference resources directly from their JavaScript modules and load them dynamically when the corresponding components are loaded.

However, the build systems for most large projects require additional optimization and refactoring of content — such as packaging and minimizing. The build system can’t execute code and predict what the result of execution will be, and there’s no reason to iterate over every possible string in JavaScript to determine if it’s a resource URL. So how do you make them “see” dynamic resources loaded by JavaScript components and include them in your build artifacts?

Custom imports in the packaging tool

One common approach is to take advantage of existing static import syntax. Some packaging tools may automatically detect formatting through file extensions, while others allow plug-ins to use custom URL schemes, such as the following example:

// Plain JavaScript import
import { loadImg } from './utils.js';

// Static resources for special "URL imports"
import imageUrl from 'asset-url:./image.png';
import wasmUrl from 'asset-url:./module.wasm';
import workerUrl from 'js-url:./worker.js';

loadImg(imageUrl);
WebAssembly.instantiateStreaming(fetch(wasmUrl));
new Worker(workerUrl);
Copy the code

When a packaging tool plug-in finds an import item with an extension or URL Scheme it recognizes (asset-URL: and js-URL: in the above example), it adds the referenced resource to the build diagram, copies it to the final destination, performs optimizations appropriate for the resource type, and returns the final URL. For use at run time.

This approach has the advantage of reusing JavaScript import syntax and ensuring that all urls are static relative paths, which makes it easy for the build system to locate this dependency.

However, it has an obvious drawback: this code doesn’t work directly in the browser, because the browser doesn’t know how to handle custom import schemes or extensions. Of course, if you have control over all the code and are already relying on packaging tools for development, that sounds fine. However, to reduce the hassle, it is becoming more common to use JavaScript modules directly in the browser (at least during development). A small demo might not need a packaging tool at all, even in production.

Common import syntax in browsers and packaging tools

If you’re developing a reusable component, you want it to work in any environment, whether it’s used directly in a browser or pre-built as part of a larger application. Most modern packaging tools accept the following JavaScript module import syntax:

new URL('./relative-path'.import.meta.url)
Copy the code

It may seem like a special syntax, but it is a valid JavaScript expression that can be used directly in a browser or statically detected and processed by a packaging tool.

Using this syntax, the previous example can be rewritten as:

// regular JavaScript import
import { loadImg } from './utils.js';
loadImg(new URL('./image.png'.import.meta.url));
WebAssembly.instantiateStreaming(
  fetch(new URL('./module.wasm'.import.meta.url)),
  { / *... * /});new Worker(new URL('./worker.js'.import.meta.url));
Copy the code

Let’s analyze how it works: new URL(…) The constructor resolves the relative URL in the first argument based on the absolute URL in the second argument. In our example, the second argument is import.meta.url, which is the URL of the current JavaScript module, so the first argument can be any path relative to it.

Its advantages and disadvantages are similar to dynamic import. Although you can use import(…) Import content, such as import(someUrl), but packaging tools specifically handle imports with static URL import(‘./some-static-url.js’) : use it as a way to preprocess known dependencies at compile time, breaking code up and loading it dynamically.

Again, you can use new URL(…) , such as New URL(relativeUrl, customAbsoluteBase), whereas new URL(‘… ‘, import.meta.url) syntax can explicitly tell the packaging tool to preprocess dependencies and package them with the main JavaScript resource.

Ambiguous relative urls

Why, you might wonder, can’t the packaging tool detect other common syntax — fetch(‘./module.wasm’) without a new URL wrapper, for example?

The reason is that, unlike import keywords, any dynamic request is parsed relative to the document itself, not relative to the current JavaScript file. For example, you have the following structure:

• Index.html:

<script src="src/main.js" type="module"></script>
Copy the code

• src/
  • main.js
  • module.wasm

If you want to load module.wasm from main.js, your first reaction might be to use a relative path reference like FETCH (‘./module.wasm’).

However, fetch does not know the URL of the JavaScript file it executes; instead, it parses the URL relative to the document. Thus, the fetch (‘)/module) wasm ‘) will try to load http://example.com/module.wasm, http://example.com/src/module.wasm, rather than expected Fail (or, worse, silently load a different resource than you expected).

By wrapping relative urls as new URLS (‘… ‘, import.meta. Url), you can avoid this problem and ensure that any supplied URL is resolved relative to the current JavaScript module’S URL (import.meta. Url) before being passed to any loader.

Simply replace fetch(‘./module.wasm’) with fetch(new URL(‘./module.wasm’, import.meta.url)) and successfully load the expected WebAssembly module, And give the packaging tool a reliable way to find these relative paths at build time.

Support in the toolchain

Packaging tools

The following packaging tools already support the new URL syntax:

• Webpack v5
• Rollup (via plugin support: @web/rollup-plugin-import-meta-assets supports generic resources, while @surma/rollup-plugin-off-main-thread supports Workers.)
• Parcel v2 (beta) Parcel V2 (beta)
• Vite

WebAssembly

When using WebAssembly, you usually don’t load the Wasm module manually, but instead import JavaScript glue code emitted by the toolchain. The toolchain below will generate a new URL for you (…) Grammar:

C/C++ compiled via Emscripten

When using the Emscripten tool chain, you can ask it to output ES6 module glue code instead of regular JS code with the following options:

$ emcc input.cpp -o output.mjs
If you don't want to use the MJS extension:
$ emcc input.cpp -o output.js -s EXPORT_ES6
Copy the code

When using this option, the output glue code will use the new URL(… , import.meta.url) syntax, so that the packaging tool can automatically find the associated Wasm files.

This syntax also supports compilation of WebAssembly threads by adding the -pthread parameter

$ emcc input.cpp -o output.mjs -pthread
If you don't want to use the MJS extension:
$ emcc input.cpp -o output.js -s EXPORT_ES6 -pthread
Copy the code

In this case, the generated Web Worker will be referenced in the same way and will also be loaded correctly by packaging tools and browsers.

Rust compiled with wASM-pack/wASM-bindgen

Wasm-pack, WebAssembly’s main Rust tool chain, also has several output modes.

By default, it outputs a JavaScript module that relies on the WebAssembly ESM integration proposal. At the time of this writing, this proposal is still experimental, and the output will only work if you use Webpack packaging.

Alternatively, you can request wASM-pack via the -target Web parameter by outputting a browser-compatible ES6 module:

$ wasm-pack build --target web
Copy the code

The output will use the new URL(… , import.meta.url) syntax, and Wasm files are automatically discovered by the packaging tool.

If you want to use WebAssembly threads with Rust, this gets a little more complicated. See the corresponding section of the guide to learn more.

In short, you can’t use arbitrary threading apis, but if you use Rayon, you can try the WASM-Bingen-Rayon adapter so that it can generate workers that run on the Web. The JavaScript glue used by wASM-Bindgen-Rayon also includes new URL(…) Syntax, so Workers can also be discovered and introduced by packaging tools.

Future import methods

import.meta.resolve

One potential future improvement is dedicated to import.meta.resolve(…) Syntax. It will allow parsing of the contents relative to the current module in a more direct manner, without the need for additional parameters.

// The current syntax
new URL('... '.import.meta.url)

// Future syntax
await import.meta.resolve('... ')
Copy the code

It also integrates better with import dependency maps and custom parsers because it is handled through the same module parsing system as the import syntax. This is also a more reliable signal for the packaging tool because it is a static syntax that does not rely on runtime apis like urls.

Import.meta. resolve has been implemented as an experimental feature in Node.js, but there are still some unanswered questions about how it should work on the Web.

Import the assertion

Import assertions are a new feature that allows you to import types other than the ECMAScript module, although only JSON types are now supported.

• foo.json

{ "answer": 42 }
Copy the code

• main.mjs

import json from './foo.json' assert { type: 'json' };
console.log(json.answer); / / 42
Copy the code

(Translator’s note: There’s something interesting about this unintuitive grammatical choiceGithub.com/tc39/propos…

They may also be used by packaging tools and replace the scenarios currently supported by new URL syntax, but types in import assertions need to be supported one by one. Currently only JSON is supported, CSS modules will be supported soon, but other types of resource imports still need a more general solution.

To learn more about this feature, check out the feature explanation on v8.dev.

summary

As you can see, there are various ways to include non-javascript resources on the web, but they have their own pros and cons, and none of them work in all toolchains at once. Some future proposals might allow us to import these resources with specialized syntax, but we are not there yet.

Until that day, new URL(… , import.meta.url) syntax is the most promising solution, and today it works in browsers, bundles, and WebAssembly toolchains.