Continuing with the previous article:Babel 1: Architecture and Principles + Practice 🔥Welcome to reprint, let more people see my article, reprint please indicate the source
In this article, we’ll take a closer look at macros — I’m sure we’re all familiar with macros, as many programmers learned their first language C/C++; Some Lisp dialects also support macros (Clojure, Scheme, for example), which I’ve heard are elegant to write; Some modern programming languages, such as Rust, Nim, Julia, Elixir, also have some support for macros. How do they solve the technical problem of implementing lisp-like macro systems? What role do macros play in these languages…
If you haven’t read the previous article, please read it first to avoid affecting your understanding of the content of this article.
The article Outlines
- On the macro
- Text substitution
- Syntactic extension
- Sweet.js
- summary
- A Plugin is a Macro
- How to write a Babel Macro
- In actual combat
- Extended information
On the macro
Macro is a batch processing term that transforms certain text patterns according to a set of predefined rules. The interpreter or compiler does this automatically when it encounters a Macro, a process called Macro Expansion. For compiled languages, macro expansion occurs at compile time, and the tool for macro expansion is often referred to as a macro expander.
You can think of a macro as code that is used to generate code, with the ability to do some parsing and code translation. Grandiosity can be divided into two types: text replacement and grammar extension
Text substitution
Macros are more or less familiar. Many programmers first learned C/C++(including objective-C, a derivative of C), and the concept of macros was introduced in C. Define a macro using the #define directive:
#define MIN(X, Y) ((X) < (Y) ? (X) : (Y))
Copy the code
If our program uses this macro, it will be expanded at compile time, for example:
MIN(a + b, c + d)
Copy the code
Will be expanded to:
((a + b) < (c + d) ? (a + b) : (c + d))
Copy the code
In addition to function macros, there are object macros in C, which we usually use to declare ‘constants ‘:
#define3.1214 PI?
Copy the code
Macros are not part of C per se, as shown above. They are provided by a C preprocessor, which produces’ real ‘C code by text substitution of source code before compilation, which is passed to the compiler.
Of course, the C preprocessor doesn’t just handle macros, it also includes header import, conditional compilation, line control, and so on
In addition, GNU M4 is a more professional/powerful/versatile preprocessor (macro expander). This is a general-purpose macro expansion that can be used not only for C, but also for other languages and text files (see this interesting article: Adding directory support for Markdown with GNU M4). See the world for more GNU M4 tutorials on M4.
Text-substitution macros are easy to understand and simple to implement because they are simply text-substitution, in other words, they act like a ‘text editor’. So this form of macros has relatively limited capabilities, such as not verifying that the syntax is valid, and it often causes problems.
So as modern programming languages have become more expressive, many of them have moved away from recommending macros/not providing macros, instead using the language’s own mechanisms (such as functions) to solve problems, which are safer, easier to understand and debug. Instead of macros, modern languages can compensate for the lack of macros by providing powerful reflection mechanisms or dynamic programming features such as Javascript proxies or Python decorators. So conversely, the reason why C language needs macros is precisely because C language is too weak.
Syntactic extension
The origin of ‘real’ macros is Lisp. This is due to some features of the language itself:
- Its syntax is very simple. onlyS-expression(Characterized by parenthesized prefix notation, s-expressions can be thought of as approximating Lisp’s abstract syntax tree (AST))
- Data is code. The S-expression itself is a tree data structure. Lisp also supports conversion between data and code
Because of the simple syntax of Lisp, there is only a thin line between data and program (quote refers to data, and no quote refers to program). In other words, programs and programs can be converted flexibly. This data-as-program, program-as-data concept makes it easy to customize macros in Lisp. Consider an example of Lisp defining a macro:
; Use defmacro to define a nonsense macro that takes a function-name argument. The macro needs to return a letter of credit
; This is a short form of the quote function, which means that the 'program' is a piece of 'data' or converts the 'program' to 'data'.
; Defun defines a function
; This is short for unquote, which translates' data 'into' program '. Unquote evaluates
; Intern converts a string to a symbol, an identifier
(defmacro nonsense (function-name) ` (defun, (intern (concat "nonsense-" function-name)) (input) ; Define a nonsense-${function-name} method
(print (concat ,function-name input)))) ; Input ` ${function - the name} ${input} `
Copy the code
If you don’t understand what the above program means, here is a Javascript implementation
Note: ‘macros’ are usually deployed at compile time, and the following code is intended to help you understand the Lisp code described above
function nonsense(name) {
let rtn
eval(`rtn = function nonsense${name}(input) {
console.log('${name}', input)
}`)
return rtn
}
Copy the code
Application macro expansion:
(nonsense "apple") ; Expand the macro, where a nonsense-apple function is created
(nonsense-apple " is good") ; Invoke the macro you just created
; => "apple is good"
Copy the code
To Lisp, a macro is a bit like a function, except that the function must return a letter of credit; When the macro is called, Lisp uses the unquote function to convert the letter data returned by the macro into the program.
From the examples above, you’ll be amazed at how neat and simple Lisp’s macro implementation is. It made me want to learn Clojure, but then I learned Elixir 😂.
Lisp macros are flexible because of their simple syntax (an S-expression can be equivalent to its AST), whereas lisp-like macros are much harder to implement for languages with complex syntax (such as Javascript). This is probably why so few modern languages provide macro mechanisms.
However, many of the technical difficulties are now being solved, and many modern languages have introduced ‘lisp-like’ macro mechanisms, such as Rust, Julia, and Javascript’s sweet.js
Sweet.js
Js and Rust are from the same school, so both macro syntax and are very close (initially). It should be noted, however, that sweet.js is officially still experimental, and Github’s last submission date was two years ago, so we haven’t seen widespread adoption on the community yet. So don’t use it in a production environment, but that doesn’t prevent us from learning the macro mechanism of a modern programming language.
We will use sweet. js to implement the nosense macro we implemented above through Lisp, which is much easier to understand by comparison:
import { unwrap, fromIdentifier, fromStringLiteral } from '@sweet-js/helpers' for syntax;
syntax nosense = function (ctx) {
let name = ctx.next().value;
let funcName = 'nonsense' + unwrap(name).value
return #`function ${fromIdentifier(name, funcName)} () {
console.log(${fromStringLiteral(name, unwrap(name).value)} + input)
}`;
};
nosense Apple
nosenseApple(" is Good") // Apple is Good
Copy the code
First, sweet.js uses the syntax keyword to define a macro, whose syntax is similar to const or let.
A macro is essentially a function that is executed at compile time. This function receives a TransformerContext Object, which you also use to get an array of Syntax objects passed in by the macro application, which eventually returns an array of Syntax objects.
What is a syntax object? The syntax object is an internal representation of the syntax in Sweet.js. You can compare the quoted data in Lisp above. In a language with complex grammar, there is no way to use such a simple sequence as letter to express the grammar, while using AST is more complex and more difficult for developers to control. Therefore, most macro implementations refer to Lisp S- expressions, take the compromise scheme, convert the passed programs into Tokens, and then assemble them into data structures similar to letter.
For example, sweet.js converts foo,bar(‘baz’, 1) to a data structure like this:
As you can see from the above figure, sweet. js parses the incoming program into nested Token sequences, a structure very similar to Lisp S- expressions. That is, for closed lexical units are stored nested, as in the example above (‘baz’, 1).
Elixir also uses a similar quote/unquote mechanism, which can be understood together
The TransformerContext implements iterator methods, so we iterate over the syntax object by calling its next(). In the end the macro must return an array of syntax objects, and sweet.js simplifies development by using a syntax similar to a string template (called a syntax template), which is eventually converted into an array of syntax objects.
Note that the embedded value of a syntax template can only be a syntax object, sequence of syntax objects, or TransformerContext.
Older versions are usedPattern matchingLike Rust syntax, I personally prefer this one, which somehow fell into disuse
macro define {
rule { $x } => {
var $x
}
rule { $x = $expr } => {
var $x = $expr
}
}
define y;
define y = 5;
Copy the code
Having said that, the design of syntactic objects like sweet.js is a key technical point for modern programming languages to get close to Lisp macros. I found that Elixir, Rust, and others use similar designs. In addition to data structure design, the macro mechanism of modern programming languages includes the following features:
1️ Hygiene
Health macros mean that variables generated within the macro do not contaminate the external scope, that is, when the macro is expanded, sweet.js will avoid variables defined within the macro and external conflicts.
For example, we create a swap macro that swaps the values of variables:
Syntax swap = (CTX) => {const a = ctx.next().value ctx.next() // take ',' const b = ctx.next(). Value return # 'let temp =' syntax swap = (CTX) => {const a = ctx.next().value ctx.next() // take ',' const b = ctx.next() ${a} ${a} = ${b} ${b} = temp `; } swap foo,barCopy the code
The expansion will output
let temp_10 = foo; // The temp variable is renamed temp_10
foo = bar;
bar = temp_10;
Copy the code
If you want to refer to external variables, that’s fine. This is not recommended, however, and macros should not assume the context in which they are being expanded:
syntax swap = (ctx) => { // ... Return # ` temp = ${a} / / do not use the let declare ${a} = ${b} ${b} = temp `; }Copy the code
2 ️ ⃣ modular
The sweet.js macro is modular:
'lang sweet.js';
/ / export macro
export syntax class = function (ctx) {
// ...
};
Copy the code
Import:
import { class } from/ '.es2015-macros';
class Droid {
constructor(name, color) {
this.name = name;
this.color = color;
}
rollWithIt(it) {
return this.name + " is rolling with "+ it; }}Copy the code
In contrast to Babel, sweet. js macros are modular/explicit. Babel You need to configure various plug-ins and options in the configuration file, especially when team project builds have uniform specifications and environments, and project build script modifications may be limited. Modular macros are part of the source code, not the build script, making them flexible for use, refactoring, and deprecation.
This is where the biggest advantage of Babel-plugin-Macros, described below, comes in. In general, we want a unified, stable build environment, and developers should focus on developing code, not how to build it. Code variability is what makes these solutions possible.
Note that macros are expanded at compile time, so user code cannot be run, for example:
let log = msg= > console.log(msg); // User code, evaluated at runtime, cannot be accessed
syntax m = ctx= > {
// Macro functions are executed at compile time
log('doing some Sweet things'); // ERROR: the variable log was not found
// ...
};
Copy the code
Sweet.js, like macros in other languages, allows you to:
- New syntactic sugar (as Sweet as sweet.js) to implement your own flavor of syntactic or some experimental language features
- Custom operators, very powerful
- Eliminate repetitive code and improve the expressive power of the language.
- .
- Don’t show off
🤕 very sorry! Sweet.js is basically dead. So play with it as a toy for now, not in a production environment. Even if it’s not dead, the nonstandard syntax of sweet.js is so alien to the existing Javascript toolchain ecosystem that it can be cumbersome to develop and debug (like Typescript).
Ultimately, the failure of Sweet.js is that the community abandoned it. Javascript is getting more expressive, versions are iterating fast, and with Babel and Typescript solutions, there’s really no reason to use sweet.js
Sweet.js papers can be found here
summary
This section is a bit rambling, covering the history and classification of macros. The final summary is this quote from Elixir’s official tutorial: Explicit is better than implicit, and Clear code is better than concise code.
With great power comes great responsibility. Macros are more powerful, more difficult to handle than normal programs, and you may need to learn and understand them at a cost, so macros are a last resort.
A Plugin is a Macro
🤓 is not over, suddenly pulled a good far, break back to the topic. Why babel-plugin-macros?
If you’re not familiar with Babel Macro, read the official documentation, and creact-React-app is already built in
It starts with create-React-app (CRA), which encapsulates all project build logic in the React-Scripts service. The benefit is that developers don’t have to worry about the details of the build, and it’s easy to update the build tool. Just update React-Scripts.
If you maintain build scripts yourself, you need to upgrade a bunch of dependencies, and if you maintain build scripts across projects, it’s even more painful.
In “Why use VUE-CLI3? The importance of tools such as CRA and VUE-CLI for team project maintenance is illustrated in the article.
CRA is strongly convention. It is prepared for you according to best practices in the React community. To protect the benefits of encapsulation, it is not recommended to manually configure Webpack, Babel… This is why babel-plugin-macros have been created
So finding a ‘zero configuration’ mechanism for Babel was the main motivation for babel-plugin-Macros.
This article confirms this motivation: Zero-config Code Transformation with Babel-plugin-Macros, which cites an important point: “Compilers are the New Frameworks”
Indeed, Babel plays an important role in modern front-end development, as more and more frameworks or libraries create their own Babel plug-ins, which are optimized at compile time to improve user experience, development experience, and runtime performance. Such as:
- Babel-plugin-lodash converts loDash imports into on-demand imports
- Babel-plugin-import, the plug-in mentioned in the previous article, also implements on-demand imports
- Babel-react -optimize static analysis of React code with some measures to optimize the runtime efficiency. For example, separate static props or components into constants
- Root-import overrides the root-based import path to a relative path
- Styled components A typical CSS-in-JS scheme, using the Babel plug-in to support server-side rendering, precompiled templates, style compression, clean up dead code, and improve the debugging experience.
- Preval preexecutes code at compile time
- Babel-plugin-graphql-tag precompiles graphQL queries
- .
Not all plug-ins in the plug-in scenarios listed above are generic, either tied to a particular framework or used to work with a particular file type or data. These non-generic plugins are the best ones to replace with Macro.
Take Preval for example. To use the plugin form, you first need to configure the plugin:
{
"plugins": ["preval"]}Copy the code
Code:
// The string passed to preval is executed at compile time
// The preval plugin looks for the preval identifier, extracts the string and executes it, and assigns the result of the execution to the greeting
const greeting = preval` const fs = require('fs') module.exports = fs.readFileSync(require.resolve('./greeting.txt'), 'utf8') `
Copy the code
Using Macro:
// First you need to explicitly import
import preval from 'preval.macro'
// Same as above
const greeting = preval` const fs = require('fs') module.exports = fs.readFileSync(require.resolve('./greeting.txt'), 'utf8') `
Copy the code
The effect is the same, but the meaning is not quite the same. What are the differences?
-
1️ clearly, Macro does not need configuration. Babelrc (of course, babel-plugin-macros needs to be installed). This is helpful for CRA tools that do not recommend configuring build scripts
-
2️ conversion from implicit to explicit. In the last section, we said “explicit is better than implicit.” You must declare that you use Macro in your source code via an import statement; Based on the plugin approach, you might not know where the preval identifier comes from. How is it applied? When is it applied? And often you need to work with other toolchains, such as ESlint, Typescript declarations, etc.
Macro is applied explicitly by code, making it more clear what purpose and when it is applied, with minimal intrusion into the source code. With the babel-plugin-Macro layer in the middle, we reduce the coupling to the build environment, making our code easier to migrate.
-
3️ Macro is easier to be realized than Plugin. Because it focuses on specific AST nodes, see below
-
4️ additionally, Macro can get better error notification when configuration error occurs
There are pros and cons, but Babel Macro certainly has some drawbacks, such as the fact that it can only be converted explicitly compared to plugins, which can make the code a bit wordy, but I personally feel that in some scenarios the benefits outweigh the disadvantages, and if you can be explicit, you can be explicit.
So Babel Macro is a Macro? What are the shortcomings of ‘orthodox’ macro mechanisms such as sweet.js?
-
First, Babel Macro must be a valid Javascript syntax. There is no support for custom syntaxes, and there are two sides to this argument. Legitimate Javascript syntaxes don’t break the existing tool collaboration chain, and if you allow users to create new syntaxes without any restrictions, they will likely clash with standard syntaxes in the future. Does a ‘macro’ that can’t, in turn, customize the syntax look awkward and not ‘powerful ‘?
-
Because it has to be a legitimate Javascript syntax, Babel Macro’s ability to implement Domain-specific languages (DSLS) is weakened
-
Furthermore, Babel Macro is not fundamentally different from the Babel Plugin. While Sweet.js provides explicit definitions and a syntax for applying macros, Babel Macro directly manipulates AST in a much more complex way, and you still need to understand some compilation principles that keep the average developer out of the way.
Babel can implement custom syntax, but you need to Fork @babel/ Parser to modify it (see this article for a closer look at Creating Custom JS Syntax with Babel). It’s a bit of a toss and not recommended
In short, Babel Macro is essentially no different from Babel Plugin, but encapsulates a layer on top of Plugin (the power of the layered architectural pattern), creating a new platform for developers to explicitly apply code transformation at the source level. So, any scenario that is suitable for explicit transformation should be done with the Babel Macro:
- Specific framework, library code conversion. Such as
styled-components
- Dynamically generate code.
preval
- Specific file, language processing. For example,
graphql-tag.macro
,yaml.macro
,svgr.macro
- . (see the awesome – Babel – macros)
How to write a Babel Macro
So how does Babel Macro work? Babel-plugin-macros requires the developer to explicitly import Macro. It will walk through all the import statements if the import source matches /[./] Macro (\.js)? $/ re will assume you are enabling Macro. For example, the following import statements all match the re:
import foo from 'my.macro'
import { bar } from './bar/macro'
import { baz as _baz} from 'baz/macro.js'
// Namespace imports are not supported
Copy the code
When the import statement is matched, babel-plugin-macros will import your specified Macro module or NPM package.
So what does a macro file contain? As follows:
const { createMacro } = require('babel-plugin-macros')
module.exports = createMacro(({references, state, babel}) = > {
/ /... Macro logic
})
Copy the code
The macro file must, by default, export an instance created by ceateMacro, and in its callback get some key objects:
babel
Just like a regular Babel plug-in, Macro can get onebabel-core
objectstate
This is also familiar, as the second parameter to the Visitor method of the Babel plug-in allows us to retrieve configuration information and save custom statereferences
Gets all references to the Macro export identifier. The last article introduced scope, and you probably haven’t forgotten the concepts of bindings and references. The following
Suppose the user uses your Macro like this:
import foo, {bar, baz as Baz} from './my.macro' // Create three bindings
// Start referencing these bindings
foo(1)
foo(2)
bar`by tagged Template`
;<Baz>by JSX</Baz>
Copy the code
You will get the references structure like this:
{
// key = 'bind' and value = 'reference array'
default: [NodePath/*Identifier(foo)*/, NodePath/*Identifier(foo)*/].// The default export is foo
bar: [NodePath/*Identifier(bar)*/].baz: [NodePath/*JSXIdentifier(Baz)*/].// Key is baz, not baz
}
Copy the code
AST Explorer also supports babel-plugin-Macros to play around with. The actual combat examples below are also suggested to explore here
You can then go through references and convert these nodes to implement the macros you want. Let’s go!
In actual combat
This time we model preval to create an eval.macro macro that executes some code at compile time. Such as:
import evalm from 'eval.macro'
const x = evalm` function fib(n) { const SQRT_FIVE = Math.sqrt(5); Return math.round (1/SQRT_FIVE * (math.pow (0.5 + SQRT_FIVE/2, n) - math.pow (0.5 - SQRT_FIVE/2, n))); } fib(20) `
// ↓ ↓ ↓ ↓ ↓ ↓
const x = 6765
Copy the code
Create the Macro file. As described in the previous section, we used createMacro to create a Macro instance, and pulled all the references from the references identifiers. Then we performed the AST conversion on these reference paths:
const { createMacro, MacroError } = require('babel-plugin-macros')
function myMacro({ references, state, babel }) {
// Get all references exported by default
const { default: defaultImport = [] } = references;
// Iterate over the reference and evaluate it
defaultImport.forEach(referencePath= > {
if (referencePath.parentPath.type === "TaggedTemplateExpression") {
const val = referencePath.parentPath.get("quasi").evaluate().value
const res = eval(val)
const ast = objToAst(res)
referencePath.parentPath.replaceWith(ast)
} else {
// Output a friendly error message
throw new MacroError('Only tag template strings are supported, such as evalm' 1 '.)}}); }module.exports = createMacro(myMacro);
Copy the code
For brevity, only tag template strings are supported in this example, but tag template strings may contain interpolated strings, for example:
hello`
hello world ${foo} + ${bar + baz}
`
Copy the code
Its AST structure is as follows:
We need to convert the TaggedTemplateExpression node to a string. Manual concatenation can be cumbersome, but each AST node’s Path object has an evaluate method that ‘statically evaluates’ the node:
t.evaluate(parse("5 + 5")) // { confident: true, value: 10 }
t.evaluate(parse(! "" true")) // { confident: true, value: false }
// ❌ two variables cannot be added, because the value of the variable exists only at runtime. Here, confident is false:
t.evaluate(parse("foo + foo")) // { confident: false, value: undefined }
Copy the code
Therefore, tag template strings like this cannot be evaluated:
evalm1 + `${foo}` // Contain variables
evalm1 + `${bar(1)}` // include function calls
Copy the code
This is the same as Typescript enums and other compiled language constants, which are evaluated at compile time. Only raw values and expressions of raw values are evaluated at compile time.
So, the above code is not robust enough, let’s optimize it to give the user a better warning if the evaluation fails:
defaultImport.forEach(referencePath= > {
if (referencePath.parentPath.type === "TaggedTemplateExpression") {
const evaluated = referencePath.parentPath.get("quasi").evaluate();
// Failed to convert label template string
if(! evaluated.confident) {throw new MacroError("Tag template string interpolation only supports raw values and raw value expressions");
}
try {
const res = eval(evaluated.value);
const ast = objToAst(res);
// Replace the calling node
referencePath.parentPath.replaceWith(ast);
} catch (err) {
throw new MacroError('evaluation failed:${err.message}`); }}else {
throw new MacroError("Only tag template strings are supported, for example: evalm '1 + 1'"); }});Copy the code
Next will execute after the value of the conversion for the AST, then replace TaggedTemplateExpression:
function objToAst(res) {
let str = JSON.stringify(res);
if (str == null) {
str = "undefined";
}
const variableDeclarationNode = babel.template(`var x = ${str}`, {}) ();// Fetch the AST of the initialization expression
return variableDeclarationNode.declarations[0].init;
}
Copy the code
This is where @babel/template comes in handy. It can parse string code into an AST, or parse directly using the parse method.
Ok, that’s basically the end of the article. This article gives an in-depth discussion of ‘macros’, from the text replacement macros of C to the dying sweet.js, and finally introduces babel-plugin-macros.
Babel Macro is essentially a Babel plug-in, but it’s modular and you have to import it explicitly to use it. Babel Macro operates directly on AST compared to ‘proper’ macros and requires you to understand compilation principles. Babel Macro can do things that ‘proper’ macros can do (such as sanitary macros). Although slightly simpler than the Babel plug-in, it’s still a bit wordy. In addition, Babel Macro can’t create new syntax, which makes it compatible with the existing tool ecosystem.
At last! Open your imagination 🧠, Babel Macro can do a lot of fun things. Check out Awesome Babel Macros. But remember, ‘Explicit is better than implicit, and clear code is better than clean code.’
As of October 10, 2019, the number of Nuggets fans has exceeded ✨2000✨. Continue to follow me and give me a thumbs up.
Extended information
- Zero-config code transformation with babel-plugin-macros
- RFC – babel-macros
- STOP WRITING JAVASCRIPT COMPILERS! MAKE MACROS INSTEAD
- Clojure Macro (1)
- Elixir Macro
- Rust of macro
- IOS thoughtful article | macro definition
- How does Elixir compile/execute code?
- Give the world one more GNU M4 tutorial (1)
- Why aren’t macro languages popular
- awesome-babel
- What is the support for macros in each programming language?
- Sweetjs related papers