Read the original


preface

Koa 2.x is currently the most popular NodeJS framework. The source code for Koa 2.0 is very compact and does not pack as much functionality as Express. So most of the functionality is provided by the Koa development team (which is also a producer of Express) and community contributors to the middleware that Koa uses to implement the NodeJS wrapper feature. It is very simple to import the middleware and call Koa’s use method to use it in the appropriate place. This allows you to implement some functionality by manipulating CTX internally, and we’ll discuss how common middleware is implemented and how we can develop a Koa middleware for use by ourselves and others.


Koa’s onion model introduction

We will not analyze the implementation principle of the Onion model too much, but mainly analyze how the middleware works based on the usage of the API and the Onion model.

// Onion model features
/ / the introduction of Koa
const Koa = require("koa");

// Create a service
const app = new Koa();

app.use(async (ctx, next) => {
    console.log(1);
    await next();
    console.log(2);
});

app.use(async (ctx, next) => {
    console.log(3);
    await next();
    console.log(4);
});

app.use(async (ctx, next) => {
    console.log(5);
    await next();
    console.log(6);
});

// Listen to the service
app.listen(3000);

/ / 1
/ / 3
/ / 5
/ / 6
/ / 4
/ / 2
Copy the code

We know that Koa’s use method supports asynchrony, so in order to ensure normal code execution according to the order of onion model, we need to make the code wait when calling next, and then continue to execute after asynchrony, so we recommend using async/await in Koa. The introduced middleware is called in the use method, so we can analyze that each Koa middleware returns an async function.


Koa – BodyParser middleware simulation

The koA-BodyParser middleware converts our POST requests and form submission query strings into objects and attaches them to ctx.request.body. It is convenient for us to use the value on other middleware or interfaces. It should be installed in advance before use.

npm install koa koa-bodyparser

Koa-bodyparser is used as follows:

// Koa - bodyParser
const Koa = require("koa");
const bodyParser = require("koa-bodyparser");

const app = new Koa();

// Use middleware
app.use(bodyParser());

app.use(async (ctx, next) => {
    if (ctx.path === "/" && ctx.method === "POST") {
        // With middleware, the ctx.request.body attribute automatically adds the data for the POST request
        console.log(ctx.request.body); }}); app.listen(3000);
Copy the code

According to the usage, we can see that the koA-BodyParser middleware actually introduces a function, which is executed in use. According to the characteristics of KOA, we infer that the koA-BodyParser function should return an async function. Here is the code for our simulated implementation.

// File: my-koa-bodyParser.js
const querystring = require("querystring");

module.exports = function bodyParser() {
    return async (ctx, next) => {
        await new Promise((resolve, reject) = > {
            // An array to store data
            let dataArr = [];

            // Receive data
            ctx.req.on("data", data => dataArr.push(data));

            // Consolidate the data and use Promise to succeed
            ctx.req.on("end", () = > {// Get the requested data type json or form
                let contentType = ctx.get("Content-Type");

                // Get the data Buffer format
                let data = Buffer.concat(dataArr).toString();

                if (contentType === "application/x-www-form-urlencoded") {
                    // If it is a form submission, the query string is converted into an object and assigned to ctx.request.body
                    ctx.request.body = querystring.parse(data);
                } else if (contentType === "applaction/json") {
                    // If it is JSON, the string object is converted into an object and assigned to ctx.request.body
                    ctx.request.body = JSON.parse(data);
                }

                // A successful callback was executed
                resolve();
            });
        });

        // Continue down
        await next();
    };
};
Copy the code

A few points to note in the code above are that the next call and why receiving data via stream, processing data, and hanging the data in ctx.request.body are in the Promise.

The first isnextWe know thatKoa 的 nextExecution, in fact, is the execution of the next middleware function, the nextuseIn theasyncFunction, to ensure that the asynchronous code will finish before the current code is executed, so we need to useawaitWait, followed by data from receiving to hangingctx.request.bodyBoth are performed in the Promise because the data is received asynchronously, and the entire process of processing the data needs to wait for the asynchronous completion before hanging the data in thectx.request.bodyGo on, you can guarantee us the nextuse 的 asyncThe function can be inctx.request.bodySo we useawaitWait for a Promise to succeed before executing itnext.


Koa-better-body Middleware simulation

Koa-bodyparser is still a bit weak on form submission because it doesn’t support file uploads. Koa-better-body makes up for this, but koA-better-body is a middleware version of KOA 1.x. The middleware for Koa 1.x is implemented using Generator functions and we need to convert koA-better-body to Koa 2.x middleware using koA-convert.

npm install koa koa-better-body koa-convert path uuid

Koa-better-body is used as follows:

// Koa-better-body
const Koa = require("koa");
const betterBody = require("koa-better-body");
const convert = require("koa-convert"); // Convert KOA 1.0 middleware to KOA 2.0 middleware
const path = require("path");
const fs = require("fs");
const uuid = require("uuid/v1"); // Generate a random string

const app = new Koa();

// Convert the KOA-Better-Body middleware from KOA 1.0 to KOA 2.0 and use the middleware
app.use(convert(betterBody({
    uploadDir: path.resolve(__dirname, "upload")}))); app.use(async (ctx, next) => {
    if (ctx.path === "/" && ctx.method === "POST") {
        // With middleware, the ctx.request.fields property automatically adds the file data requested by post
        console.log(ctx.request.fields);

        // Rename the file
        let imgPath = ctx.request.fields.avatar[0].path;
        letnewPath = path.resolve(__dirname, uuid()); fs.rename(imgPath, newPath); }}); app.listen(3000);
Copy the code

The main function of koA-better-body in the above code is to save the file uploaded by the form to a locally specified folder and hang the file stream object on the ctx.request.fields property. Next, we will simulate the functionality of KOA-Better-Body to implement a version of middleware based on KOA 2.x to handle file uploads.

// File: my-koa-better-body.js
const fs = require("fs");
const uuid = require("uuid/v1");
const path = require("path");

// The Buffer extension split method is prepared for later use
Buffer.prototype.split = function (sep) {
    let len = Buffer.from(sep).length; // The number of bytes in the delimiter
    let result = []; // The array returned
    let start = 0; // Find the starting position of Buffer
    let offset = 0; / / the offset

    // Loop to find the delimiter
    while ((offset = this.indexOf(sep, start)) ! = =- 1) {
        // Cut off the part before the delimiter and store it
        result.push(this.slice(start, offset));
        start = offset + len;
    }

    // Handle the rest
    result.push(this.slice(start));

    // Return the result
    return result;
}

module.exports = function (options) {
    return async (ctx, next) => {
        await new Promise((resolve, reject) = > {
            let dataArr = []; // Store the read data

            // Read data
            ctx.req.on("data", data => dataArr.push(data));

            ctx.req.on("end", () = > {// Get the separator string for each segment of the request body
                let bondery = `--${ctx.get("content-Type").split("=") [1]}`;

                // Get newlines for different systems
                let lineBreak = process.platform === "win32" ? "\r\n" : "\n";

                // Final return result for non-file type data
                let fields = {};

                // Remove useless headers and tails, i.e., the leading '' and the trailing '--'.
                dataArr = dataArr.split(bondery).slice(1.- 1);

                // Loop through the contents of each Buffer in the dataArr
                dataArr.forEach(lines= > {
                    // For normal values, the information consists of a line containing the key name + two newlines + data values + newlines
                    // For a file, the information consists of a line containing filename + two newlines + file content + newlines
                    let [head, tail] = lines.split(`${lineBreak}${lineBreak}`);

                    // Determine if it is a file. If it is a file, create a file and write it. If it is a normal value, store it in the Fields object
                    if (head.includes("filename")) {
                        // To prevent file contents from being split with a newline, re-intercept the contents and remove the last newline
                        let tail = lines.slice(head.length + 2 * lineBreak.length, -lineBreak.length);

                        // Create a writable stream and specify the path to write to: absolute path + specified folder + random file name, and finally write to the file
                        fs.createWriteStream(path.join(__dirname, options.uploadDir, uuid())).end(tail);
                    } else {
                        // Is a normal value to retrieve the key name
                        let key = head.match(/name="(\w+)"/) [1];

                        // Set key to fields tail to remove the trailing newline
                        fields[key] = tail.toString("utf8").slice(0, -lineBreak.length); }});// Hang the processed fields object on ctx.request.fields and complete the Promise
                ctx.request.fields = fields;
                resolve();
            });
        });

        // Execute down
        awaitnext(); }}Copy the code

The above content logic can be understood through code comments, which is to simulate the functional logic of KOA-better-body. Our main concern lies in the way the middleware is implemented. The asynchronous operation of the above functionality is still to read data, which is still executed in the Promise to wait for the completion of data processing. And waits with await, the Promise performs a successful call to Next.


Koa-views Middleware simulation

Node templates are the tools we often use to help us render pages on the server side. There are many kinds of templates, so koA-View middleware appeared to help us to accommodate these templates, and install the dependent modules first.

npm install koa koa-views ejs

Here is an EJS template file:

<! -- file: index.ejs -->

      
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>ejs</title>
</head>
<body>
    <%=name%>
    <%=age%>

    <%if (name= ="panda") {% >
        panda
    <%} else{% >
        shen
    <%} %>

    <%arr.forEach(item= >{% ><li><%=item%></li>
    <% %})>
</body>
</html>
Copy the code

Koa-views:

// koa-views
const Koa = require("koa");
const views = require("koa-views");
const path = require("path");

const app = new Koa();

// Use middleware
app.use(views(path.resolve(__dirname, "views"), {
    extension: "ejs"
}));

app.use(async (ctx, next) => {
    await ctx.render("index", { name: "panda".age: 20.arr: [1.2.3]}); }); app.listen(3000);
Copy the code

It can be seen that after we use KOA-Views middleware, the render method on CTX helps us to achieve the rendering and response page of the template. It is the same as using EJS’s own render method directly, and it can be seen from the usage that the render method is executed asynchronously. So we need to wait with await. Next we will simulate implementing a simple version of KOA-Views middleware.

// File: my-koa-views.js
const fs = require("fs");
const path = require("path");
const { promisify } = require("util");

// Convert the file-reading method to a Promise
const readFile = promisify(fs.radFile);

// Middleware everywhere
module.exports = function (dir, options) {
    return async (ctx, next) => {
        // Dynamically introduce template-dependent modules
        const view = require(options.extension);

        ctx.render = async (filename, data) => {
            // Read file contents asynchronously
            let tmpl = await readFile(path.join(dir, `${filename}.${options.extension}`), "utf8");

            // Render the template and return the page string
            let pageStr = view.render(tmpl, data);

            // Set the response type and respond to the page
            ctx.set("Content-Type"."text/html; charset=utf8");
            ctx.body = pageStr;
        }

        // Continue down
        awaitnext(); }}Copy the code

The render method attached to CTX is executed asynchronously because the internal reading of the template file is executed asynchronously and requires waiting. Therefore, the render method is async function, which dynamically introduces the templates we use inside the middleware, such as EJS, Use the corresponding render method inside ctx.render to retrieve the page string after the replacement data and respond with the HTML type.


Koa – Static middleware simulation

The following is the usage of the KOA-static middleware. The code used depends on the following, which must be installed before use.

npm install koa koa-static mime

Koa-static = koa-static

// koa-static
const Koa = require("koa");
const static = require("koa-static");
const path = require("path");

const app = new Koa();

app.use(static(path.resolve(__dirname, "public")));

app.use(async (ctx, next) => {
    ctx.body = "hello world";
});

app.listen(3000);
Copy the code

Through use and analysis, we know that the function of KOA-static middleware is to help us process static files when the server receives a request. If we directly access the file name, we will look for this file and respond directly. If there is no such file path, we will treat it as a folder and look for the index.html under the folder. If it does, it responds directly; if it does not, it is handed over to other middleware.

// File: my-koa-static.js
const fs = require("fs");
const path = require("path");
const mime = require("mime");
const { promisify } = require("util");

// Convert stat and access to promises
const stat = promisify(fs.stat);
const access = promisify(fs.access)

module.exports = function (dir) {
    return async (ctx, next) => {
        // Handle the access route as an absolute path, using join because it might be /
        let realPath = path.join(dir, ctx.path);

        try {
            // Get the stat object
            let statObj = await stat(realPath);

            // If it is a file, set the file type and respond directly to the content, otherwise look for index.html as a folder
            if (statObj.isFile()) {
                ctx.set("Content-Type".`${mime.getType()}; charset=utf8`);
                ctx.body = fs.createReadStream(realPath);
            } else {
                let filename = path.join(realPath, "index.html");

                // If the file does not exist, execute next in catch to other middleware
                await access(filename);

                // Set the file type and respond to the content
                ctx.set("Content-Type"."text/html; charset=utf8"); ctx.body = fs.createReadStream(filename); }}catch (e) {
            awaitnext(); }}}Copy the code

In the above logic, we need to check whether the path exists. Since the functions we export are async functions, we convert stat and access into promises and use try… Catch is used to catch, and when the path is illegal, next is called and handed over to other middleware.


Koa-router middleware simulation

In the Express framework, routing is built into the framework, whereas in Koa it is not built into the framework. It is implemented using the KOA-Router middleware and needs to be installed before use.

npm install koa koa-router

The KOA-Router is very powerful, so we will use it briefly and simulate it based on the functionality used.

// A simple koa-router
const Koa = require("Koa");
const Router = require("koa-router");

const app = new Koa();
const router = new Router();

router.get("/panda", (ctx, next) => {
    ctx.body = "panda";
});

router.get("/panda", (ctx, next) => {
    ctx.body = "pandashen";
});

router.get("/shen", (ctx, next) => {
    ctx.body = "shen";
})

// Call the routing middleware
app.use(router.routes());

app.listen(3000);
Copy the code

It can be seen from the above that koa-Router exports a class. When using it, it needs to create an instance and call the routes method of the instance to connect the async function returned by the method. However, when matching the route, it will match the path in the route GET method and execute the internal callback function in serial. When all the callback functions are completed, the entire Koa serial next is executed. The principle is the same as other middleware, and I will briefly implement the function used above.

// File: my-koa-router.js
// Controls each routing layer class
class Layer {
    constructor(path, cb) {
        this.path = path;
        this.cb = cb;
    }
    match(path) {
        // If the route of the address is equal to the currently configured route, return true, otherwise return false
        return path === this.path; }}// Route class
class Router {
    constructor() {
        {path: / XXX, fn: cb}
        this.layers = [];
    }
    get(path, cb) {
        // Store the route object into an array
        this.layers.push(new Layer(path, cb));
    }
    compose(ctx, next, handlers) {
        // Execute the matching routing functions in series
        function dispatch(index) {
            // If the current index is greater than the length of the stored route object, execute Koa's next method
            if(index >= handlers.length) return next();

            // Otherwise call the callback execution of the fetched route object and pass in a function that recursively dispatches (index + 1) from the passed function
            // The purpose is to execute the callback function on the next routing object
            handlers[index].cb(ctx, () => dispatch(index + 1));
        }

        // Execute the route object's callback function for the first time
        dispatch(0);
    }
    routes() {
        return async (ctx, next) { // Next is currently Koa's own next, that is, Koa's other middleware
            // Select a route with the same path
            let handlers = this.layers.filter(layer= > layer.match(ctx.path));
            this.compose(ctx, next, handlers); }}}Copy the code

We created a Router class and defined the get method, post, etc. We just implemented get. The logic in GET is to build the parameter function that calls the get method and the route string into an object and store it in the array layers. Therefore, we create Layer class specially for constructing route objects, which is convenient for extension. When routing matches, we can get the route string according to ctx.path, filter the route object in the array that does not match the route, call the compose method, and pass in the filtered array as parameter Handlers. Executes the callback function on the route object serially.

composeThe implementation idea of this method is very important inKoaSource code used in tandem middleware, inReactSource code used in seriesredux 的 promise,thunk 和 loggerSuch modules, our implementation is a simple version, and there is no compatible asynchronous, the main idea is recursiondispatchThe callback function is executed each time the callback function of the next route object in the array is fetched until the callback function of all matched routes is executedKoaThe next middlewarenextNotice herenextArguments that are different from the callback function in an arraynext, the callback function of the routing object in the arraynextRepresents the callback for the next matched route.


conclusion

We have analyzed and simulated some middleware above. In fact, we can understand the advantages of Koa compared with Express is that it is not so heavy, easy to develop and use, and all the required functions can be realized by the corresponding middleware. Using middleware can bring us some benefits. For example, we can mount the processed data and new methods on CTX, which is convenient for use in the callback function passed by use later. It can also help us to process some common logic, so that we will not process it in every callback of USE, which greatly reduces the redundant code. From this point of view, the process of using middleware for Koa is a typical “decorator” pattern. After the above analysis, I believe that you also understand Koa’s “Onion model” and asynchronous characteristics, and know how to develop their own middleware.