Finally use AST in demand hot!
AST and I
You can read about AST concepts and recast libraries used to manipulate AST trees in groups
Abstract Syntax Tree (AST)
At that time, I felt empty after sharing it. Although I understood some basic concepts and made a small demo, it was still too superficial for practical application. As it happens, AST has been used twice recently. My experience:
TMD really cool!
Enumeration library and JSDOC collision
When the team finished compiling the enumeration of the project and encapsulated it into a library, MR sent it over.
Instead of maintaining a copy for each project, the previously scattered enumerations are maintained in a private library.
But the documentation seems a bit much, dozens of JS scripts. so
Me: there should be another document that tells developers what these enumerations do, so they don’t have to find the code to call it.
The team said yes, and then investigated the JSDOC library and found that the code format did not meet the requirements of JSDOC. For example: remarks like the former, not the latter: /** this is a JSDOC recognized note */ /* this is a JSDOC recognized note */
For example, export a type like the former, not the latter:
const applyTypeObj = {
/** Ordinary delivery */
NORMAL_APPLY: 0./** single-click delivery */
ONE_CLICK_APPLY: 1./** By invitation */
INVITE_APPLY: 2
}
export const applyTypeEnum = Object.freeze(applyTypeObj)
Copy the code// Common delivery
export const NORMAL_APPLY = 0
// Single-click delivery
export const ONE_CLICK_APPLY = 1
// By invitation
export const INVITE_APPLY = 2
Copy the codeWith 21 files and a lot of enumerations in them, you can figure out what the labor cost is, and it’s boring, tedious, error-prone.
Why don’t you ask the amazing AST
Given the experience we shared last time, it should be easy to write such conversion code this time. Think about the basic process first
Recursively read project files -> Read files -> AST operations -> Write files
The core of this process is the AST operation. Still using our lovely Recast library.
function recastFileName(path, fileName) {
fs.readFile(path, function(err, data) {
// Failed to read the file
if (err) {
throw err
}
const code = data.toString()
console.log(code)
const ast = recast.parse(code)
let i = 0
// The simple thing to do is to consolidate all the var definitions and values that are Literal
const maps = {}
// The remarks for each field exist here
const markMap = {}
let markDown = ' '
recast.visit(ast, {
visitExportNamedDeclaration: function(path) {
const init = path.node.declaration.declarations[0].init
const key = path.node.declaration.declarations[0].id.name
let value = init.value
const type = init.type
if (type === 'UnaryExpression') {
value = eval(`${init.operator}${init.argument.value}`)}if (type === 'Literal' || type === 'UnaryExpression') {
maps[key] = value
path.node.comments &&
path.node.comments.map(item= > {
markDown = `/**
* Enum for ${fileName}
* ${item.value}
* @enum {number}
*/\n`
})
return null
}
return false
},
visitVariableDeclaration: function(path) {
if(! path.value.declarations || ! path.value.declarations[0) | |! path.value.declarations[0].init.elements
) {
return false
}
console.log('define')
console.log(path.value.declarations[0].init.elements)
path.value.declarations[0].init.elements.map(element= > {
const key = element.properties[0].value.name
const value = element.properties[1].value.value
element.properties[0].value = memberExpression(id(`${fileName}Obj`), id(key))
markMap[key] = value
})
return false}})if (!Object.keys(maps).length) {
console.log('No conversion required')
return
}
let mapString = '{\n'
Object.keys(maps).map((key, index) = > {
if (markMap[key]) mapString += ` / * *${markMap[key]} */\n`
if (index === Object.keys(maps).length - 1) {
mapString += `"${key}": ${maps[key]}\n`
} else {
mapString += `"${key}": ${maps[key]},\n`
}
})
mapString += '} '
const res = `const ${fileName}Obj = ${mapString}\nexport const ${fileName}Enum = Object.freeze(${fileName}Obj)\n`
const output = res + recast.print(ast).code
const finel = recast.print(recast.parse(output)).code
console.log(finel)
console.log(output)
fs.writeFile(path, `${markDown}\n${finel}`, {}, function() {
console.log(`wirte ${fileName}OK! `)})})}Copy the codeThe code looks stupid, so I’ll just cover the core code.
const map = []({
// It is easy to see that this method is used to capture the export statement
visitExportNamedDeclaration: function(path) {
const init = path.node.declaration.declarations[0].init
const key = path.node.declaration.declarations[0].id.name
let value = init.value
const type = init.type
Map: {NORMAL_APPLY: 0} */
if (type === 'Literal' || type === 'UnaryExpression') {
maps[key] = value
}
return false}})Copy the codeWhen we have the enumeration stored in the map, we can splice the map and stuff it into the code file.
let mapString = '{\n'
Object.keys(maps).map((key, index) = > {
if (markMap[key]) mapString += ` / * *${markMap[key]} */\n`
if (index === Object.keys(maps).length - 1) {
mapString += `"${key}": ${maps[key]}\n`
} else {
mapString += `"${key}": ${maps[key]},\n`
}
})
mapString += '} '
writeFile(mapString)
Copy the codeOf course, there are a lot of things to be aware of, such as uniform remarks in the header and separate remarks in the enumeration. I won’t go into it here. You can flip to the top and peruse it.
That requirement worked out pretty well.
One more need
Modify applets routing parameters
When I finished the transformation of the enumeration library for half a day, I was a little excited, and happened to have a requirement similar to the one above.
Small program project, the route jump looks like this:
wx.navigateTo({
url: `/pages/resumeOptimize? jobId=The ${this.jobId}&resume_enhance_source=apply_work_success&workid=The ${this.jobId}&service_type=resume_optimization`
})
Copy the codeLong, ugly, late add parameter prone to error.
So I have to write it like this.
wx.navigateTo({
url: `/pages/resumeOptimize?${qs.stringify({
jobId: this.jobId,
resume_enhance_source: 'apply_work_success',
workid: this.jobId,
service_type: 'resume_optimization'
})} `
})
Copy the codeElegant, beautifully indented and easy to maintain.
Or the amazing AST?
This one is definitely harder than the last one. First, the code file is not js, but.wpy.
Because the small program uses the WEPY framework, like vUE structure.
<template></template>
<script></script>
<style></style>
Copy the codeThe first step is to extract the script separately from the structure. With powerful regex, of course.
function getScript(code) {
let jsReg = /<script>[\s|\S]*?<\/script>/ig;
const scriptColletion = code.match(jsReg)[0].replace(/<script>/, ' ').replace(/<\/script>/.' ');
return scriptColletion
}
Copy the codeIt’s easy to get the script content. Assuming we’re done with the AST, we’re going to save the code file back, same thing.
Function setScript(code, script) {let jsReg = /<script>[\s|\S]*?<\/script>/ig;
return code.replace(jsReg, `<script>\n${script}</script>`)}Copy the codeDemand analysis
The last section was just a simple implementation of code access, and now comes the big part. Let’s first analyze what we’re going to do.
- Intercept code
wx.navigateTo或wepy.navigateToThese two apis are the same and can be called by developers - Replace the API argument with the template string
qs.stringifyThe method call - If the file has a second step, and the file header does not
import qs from 'qs', need to add manually
Intercept API
The API method call is obviously an ExpressionStatement, so we simply intercepted it, and everything else is done in the visitExpressionStatement callback.
recast.visit(ast, { visitExpressionStatement: function(path) { const callee = path.node.expression.callee if(! callee || ! Callee.object) {return false} const objName = callee.object.name const fnName = callee.property.name // Caller is WX or wepy If (objName = = = 'wx' | | objName = = = 'wepy') {/ / jump if (fnName = = = 'navigateTo') {/ / interception to}}}, return false}Copy the codeTemplate string replacement
The origin of evil dreams
Here’s the core function, which took me more than half a day. We are in the ‘intercepted’ position of the above code. The syntax tree structure of Wx. NavigateTo is found using devTool. An 🌰 :
wepy.navigateTo({
url: `/pages/detail/jobDetail? id=${e.id}&from=job_detail&num=${e.index}&uniqueKey=${uniqueKey}`
})
Copy the code<pre>if(fnName === 'navigateTo') { const argument = path.node.expression.arguments[0] let {expressions, quasis} = argument.properties[0].value if(! expressions || ! quasis || ! expressions.length) { return false } if(expressions.length< 2) {return false}
expressions = expressions.map((val)= >{ const res = recast.print(val) return res.code }) let url = '' quasis = quasis.map((val) => { const path = val.original.value.cooked if(/\? /.test(path)) {// save url url = path.split('? ')[0] return path.split('? ')[1] } return path }) }</pre>
Copy the codeIn fact, the AST splits this line of code into two groups, expressions and Quasis, which I’ll print out. The former is an expression and the latter is a string. Exactly the length of the expression equals the length of the string – 1. This conforms to the format of the template string.
["e.id"."e.index"."uniqueKey"]
["id="."&from=job_detail&num="."&uniqueKey=".""]
Copy the codeI need to concatenate the two arrays and quote the string.
<pre>const express = assignArray(quasis, expressions).join('').split('&') function assignArray(arr2, Arr2 = arr2.map((val) => val.split('&').map((equel) => {if(! equel || ! equel.split('=')[1]) { return equel } const value = '\'' + equel.split('=')[1] + '\'' return [equel.split('=')[0], value].join('=') }).join('&')) arr1.forEach((item, index) => { arr2.splice(2 * (index + 1) - 1, 0, item) }) return arr2 }</pre>
Copy the code[“id=e.id”, “from=’job_detail'”, “num=e.index”, “uniqueKey=uniqueKey”] Next, convert the above array to the parameters of the function
const results = giveQsString(express, url)
function giveQsString(expressArr, url) {
let str = `url: \`${url}? \${qs.stringify({\n`
expressArr.map((val, index) = > {
const [key, value] = val.split('=')
if(index === expressArr.length - 1)
str += ` ${key}: ${value}\n`
else
str += ` ${key}: ${value},\n`
})
str += `})} \ ` `
console.log(str)
return str
}
Copy the codeSo it ends up looking like this.
/* url: `/pages/detail/jobDetail? ${qs.stringify({ id: e.id, from: 'job_detail', num: e.index, uniqueKey: uniqueKey })}` */
Copy the codeThe most important step is to fill this parameter into the Navigate method
path.node.expression.arguments[0].properties[0] = templateElement({
"cooked": results, "raw": results
}, false)
Copy the codeAt this point, the CORE AST is complete.
Combined with qs
If the file does the above steps, it needs to import QS, which is checked by the ImportDeclaration. If the QS module is not imported, the downstream is told to append import statements to the header of the file.
visitImportDeclaration: Function (path) {if(path.node.source.value === 'qs') {needQs = false} return false}Copy the codeconclusion
Let me conclude with a quote from my weekly paper
AST was used in both requirements this week, and it was the first time that AST was used in the actual project. The two requirements produced different effects. Code structure and comment changes: the code file format of this library is relatively uniform, the number of files is more, it is not suitable for manual rewriting one by one, using AST to reduce a lot of time, and the difficulty is not high
Qs route parameters: the code format of this library is WEPY type, which is quite difficult to do. I spent a whole day writing AST code, and finally realized the global route replacement. After the replacement, I found that there are not many files to replace, so this is a counter example. Therefore, before deciding to use an AST, it is important to research whether it is suitable for use. I think the following two conditions should be met: 1. AST code is easy to write. 2. Heavy workload of repetition
AST cow force!