Template engine is usually used to dynamically generate Web pages in Web development tools, such as Smarty commonly used in PHP, Jinja of Python, Jade of Node, etc. This article will learn how to write a template engine using two simple template engine projects: Approach: Building a Toy Template Engine in Python and JavaScript Micro-templating.

A general template consists of the following three parts:

Variables and code blocks are usually identified by specific delimiters, such as:

Hello, !  {% if role == "admin" %}Dashboard  {% end %}Copy the code

Rendering text returns the text itself; Rendering of variables and chunks depends on the values we give variable names and the conventions of the chunking syntax (conditions, loops, etc.). To evaluate a string as a variable, the first thing that comes to mind is the eval method:

name = "rainy" print("Hello, " + eval("name") + "!" )# Hello, rainy!Copy the code

The eval method in many programming languages is used to convert strings into expressions for evaluation, doing much like the compiler itself, whereas the template engine is essentially a compiler for templates. We know that compilers generally use abstract syntax tree (AST) structure to represent program source code. If we regard templates as source code, we can also represent them as abstract syntax tree. For example, the above template file can be expressed as:

To turn a template file into the AST structure shown above, you first need to partition it by delimiters, for example in Python:

import reVAR_TOKEN_START = '  VAR_TOKEN_END = '  BLOCK_TOKEN_START = '{%'  BLOCK_TOKEN_END = '%}'TOK_REGEX = re.compile(r"(%s.*?%s|%s.*?%s)" % (      VAR_TOKEN_START,    VAR_TOKEN_END,    BLOCK_TOKEN_START,    BLOCK_TOKEN_END))content = """Hello, !  {% if role == "admin" %}Dashboard  {% end %}"""TOK_REGEX.split(content)# OUTPUT =>['Hello, ', '', '\n', '{% if role == "admin" %}', '\nDashboard\n', '{% end %}', '']Copy the code

After building the AST, render each node one by one. For example, render variables can be used in the following way:

def resolve(name, context):      for tok in name.split('.'):        context = context[tok]    return contextclass VarTmpl():      def __init__(self, var):        self.var = var    def render(self, **kwargs):        return resolve(self.var, kwargs)tmpl = VarTmpl("name")tmpl.render(name = "rainy")     #=> rainy  tmpl.render(name = "python")    #=> python  Copy the code

Rendering blocks is a little more complicated but similar in principle to Eval:

role = 'user'  eval('role == "admin"')  # OUTPUTFalse  Copy the code

But all the block syntax and evaluation rules need to be redefined, interested can view the source code. Now let’s look at a solution based on Js.

As you can see, the core of a template engine is to distinguish between strings and expressions, which themselves are rendered as strings. To switch between strings and expressions, the Python version above uses eval (or, more technically, ast.literal_eval). There is, of course, a similar eval method in Js, but Js has another very flexible feature. When defining a function, you can do it in one of the following ways:

var Tmpl = function(context){ with(context){ console.log(name); }}Tmpl({name: "rainy"}); //=> rainyvar raw = "name"; var Tmpl = new Function("context", "with(context){console.log("+ raw+ "); } "); Tmpl({name: "rainy"}); //=> rainy Tmpl({name: "js"}); //=> jsCopy the code

That is to say, we can transform string into expression through the method of new Function(). Combined with the steps of split-evaluation-reorganization mentioned above, we will look at the simplified version of John Resig:

(function(){ this.tmpl = function tmpl(str, data){ var fn = new Function("obj", "var p=[];" + "with(obj){p.ush ('" + str.replace (/[\r\t\n]/g, "")// .split("<%").join("\t") .replace(/\t=(.*?) %>/g, "',$1,'") .split("\t").join("');" ) .split("%>").join("p.push('") + "'); }return p.join('');" ); return data ? fn( data ) : fn; }; }) (); console.log(tmpl("Hello, <%=name%>!" , {name: "rainy"})); // OUTPUT"Hello, rainy!"Copy the code

In this (miniaturized) template engine less than 15 lines, the template is first split according to the agreed delimiter:

var str = "Hello, <%=name%>!" ; str = str.split("<%").join("\t"); //=> 'Hello, \t=name%>! ' str = str.replace(/\t=(.*?) %>/g, "',$1,'"); //=> 'Hello, \',name,\'! 'Copy the code

Note that this line is in the definition of new Function(), equivalent to:

function fn(str, data){ var p = []; with(data){ p.push('Hello, ',name,'! '); // p === ['Hello, ', name, '!']; }; }Copy the code

With (data){} name === data.name

p === ['Hello, ', 'rainy', '!']; p.join('') === "Hello, rainy!" ;Copy the code

That’s the core of the mini-template engine. If you need to handle single quotes, you can add them to the STR handler:

STR. Replace (/ / \ r \ t \ n/g, ""). The replace (/ '/ g,"/r ") / / all single quotation marks with \ r. split (" < % "). The join (" \ t "). The replace (/ \ t = (. *?) %>/g, "',$1,'") .split("\t").join("');" ). The split (" % > "). The join (" p.p ush ('). "the replace (/ \ r/g," \ \ "") / / replacement receipt within quotation marksCopy the code

conclusion

The ostensible complexity of a template engine is the construction and manipulation of abstract syntax trees, but the core problem lies in the distinction between variable names and values, that is, between programs and data. Interestingly, in Lisp, “data is program, program is data”, there is no essential difference between them. You can read this article: The Nature of Lisp. Template engines are very useful, and exploring them from a practical point of view can lead to other areas, which is the greatest joy of Programming 😀

reference