Handwritten express.js source code

In the last article we looked at how to write a Web server using the Node.js native API. The code is ugly, but the basic functionality is still there. But instead of writing directly in native apis, we use frameworks such as Express for this article. As we’ve seen in the last article, we can guess that There’s no dark magic in Express, and that it’s just a wrapper around the native API, designed to provide better extensibility, easier use, and more elegant code. As usual, this article will start with the basic use of Express, and then handwritten Express to replace it, that is, the source code parsing.

The code has been uploaded to GitHub, so it is better to play with the code while reading the article:Github.com/dennis-jian…

A simple example

Using Express to create a simple Hello World can be done in a few lines of code.

const express = require('express');
const app = express();
const port = 3000;

app.get('/'.(req, res) = > {
  res.send('Hello World! ');
});

app.listen(port, () = > {
  console.log(`Example app listening at http://localhost:${port}`);
});
Copy the code

As you can see, Express routes can be handled directly with the app.get method, which is much more elegant than writing a bunch of ifs in http.createserver. Let’s rewrite the code from the previous article in this way:

const path = require("path");
const express = require("express");
const fs = require("fs");
const url = require("url");

const app = express();
const port = 3000;

app.get("/".(req, res) = > {
  res.end("Hello World");
});

app.get("/api/users".(req, res) = > {
  const resData = [
    {
      id: 1.name: "Xiao Ming".age: 18}, {id: 2.name: "Little red".age: 19,},]; res.setHeader("Content-Type"."application/json");
  res.end(JSON.stringify(resData));
});

app.post("/api/users".(req, res) = > {
  let postData = "";
  req.on("data".(chunk) = > {
    postData = postData + chunk;
  });

  req.on("end".() = > {
    // Insert content into db.txt after data transfer
    fs.appendFile(path.join(__dirname, "db.txt"), postData, () = > {
      res.end(postData); // Write the data and return it again
    });
  });
});

app.listen(port, () = > {
  console.log(`Server is running on http://localhost:${port}/ `);
});

Copy the code

Express also supports middleware, which we write to print out the path of each request:

app.use((req, res, next) = > {
  const urlObject = url.parse(req.url);
  const { pathname } = urlObject;

  console.log(`request path: ${pathname}`);

  next();
});
Copy the code

Express also supports static resource hosting, but its API needs to specify a separate folder to store static resources, for example, we create a public folder to store static resources, using Express. Static middleware configuration:

app.use(express.static(path.join(__dirname, 'public')));
Copy the code

Then you can get static resources:

Handwritten source

Handwritten source code is the focus of this article, the front is just paving the way, the goal of this article is to write an Express to replace the express API used in front, in fact, the source code parsing. Before we begin, let’s take a look at what apis are used:

express()The first one must beexpressFunction, which returns one when it runsappThis is the case with many of the following methodsappOn the.

app.listenThis method is similar to nativeserver.listenTo start the server.

app.get, this is the API that handles routing, and things like thatapp.postAnd so on.

app.useThis is the call entry for middleware, and all middleware is called through this method.

express.staticThis middleware helps us do static resource hosting, which is actually another library calledserve-staticBecause withExpressArchitecture matters little, this article will not talk about his source code.

All the handwritten codes in this article are written according to the official source code. The method name and variable name should be consistent with the official source code as far as possible. You can compare them and see that WHEN I write the specific method, I will post the official source code address.

express()

The first thing we need to write is express(), which is where it all starts. It creates and returns an app, which is our Web server.

// express.js
var mixin = require('merge-descriptors');
var proto = require('./application');

// Create a web server method
function createApplication() {
  // This app method is actually a callback passed to http.createserver
  var app = function (req, res) {}; mixin(app, proto,false);

  return app;
}

exports = module.exports = createApplication;
Copy the code

The code above is the code we execute when we run Express (), which is an empty shell. The app returned is temporarily an empty function. The real app is not here, but on proto, which is actually application.js. We then assign everything on proto to app with the following line:

mixin(app, proto, false);
Copy the code

This line uses a third party library, merge-descriptors. There are no more than a few lines of code and the simple task is to assign the merge-descriptors attributes to app. For those interested in the merge-descriptors source, see here: Github.com/component/m… .

The reason Express uses mixins here, rather than normal object-oriented inheritance, is that in addition to mixin proto, it also needs other mixin libraries, that is, multiple inheritance, which I omit here, but the official source code is available.

Express. js can be found at github.com/expressjs/e…

app.listen

Express.js is just a shell. The real app is inside application.js, so app.listen is also there.

// application.js

var app = exports = module.exports = {};

app.listen = function listen() {
  var server = http.createServer(this);
  return server.listen.apply(server, arguments);
};
Copy the code

The above code calls the native HTTP module to create a server, but passes this. What is this? Recall that we used express like this:

const app = express();

app.listen(3000);
Copy the code

So the actual caller to listen is the return value of express(), which is the return value of createApplication in express.js above, which is this function:

var app = function (req, res) {};Copy the code

So this is also the function, so I annotated it in express.js. This function is the callback to http.createserver. Now that this function is empty, it should actually be the processing entry to the entire Web server, so let’s add processing logic to it and add a line of code to it:

var app = function(req, res) {
  app.handle(req, res);    // This is the real server processing entry
};
Copy the code

app.handle

App.handle is also mounted under app, so it is actually inside the application.js file. Here’s what it does:

app.handle = function handle(req, res) {
  var router = this._router;

  // The final processing method
  var done = finalhandler(req, res);

  // If the router is not defined
  // End to return
  if(! router) { done();return;
  }

  // If there is a router, use the router to process it
  router.handle(req, res, done);
}
Copy the code

As you can see from the above code, the router is actually handling the route. This is an instance of the Router mounted on this, and we haven’t assigned it a value yet. If not, we will run finalHandler and finish processing. Finalhandler is also a third-party library that GitHub links to: github.com/pillarjs/fi… . The function of this library is not complicated, it is to help you handle some finishing work, such as all routes are not matched, you may need to return 404 and record error log, this library can do for you.

app.get

App. router = app. router = app. router = app. router = app. router = app. router = app. router = app. router In fact, app._router is assigned in several places. One place is in the HTTP verb processing method, such as app.get or app.post. Both app.get and app.post are handled by calling the Router method, so you can write all of these methods in a loop.

// HTTP verb methods
var methods = ['get'.'post'];
methods.forEach(function (method) {
  app[method] = function (path) {
    this.lazyrouter();

    var route = this._router.route(path);
    route[method].apply(route, Array.prototype.slice.call(arguments.1));
    return this; }});Copy the code

The above code HTTP verbs are placed in an array, the official source of the array is also a third-party library maintenance, the name of the array is called methods, GitHub address here: github.com/jshttp/meth… . My example, because I only need two verbs, simplifies to just using arrays. This code actually creates a function for the app with the same name as each verb. All verbs are handled by the same method, which calls the corresponding method in the Router. This abstraction of the different parts of the code to reuse the common parts is a bit like the design pattern I wrote about in another article —- Share pattern.

Note that in addition to calling the router to handle the route, the above code has one line:

this.lazyrouter();
Copy the code

The lazyRouter method is where we assign a value to this._router. The code is simple: check if _router exists, and if it does not, assign it an instance of Router:

app.lazyrouter = function lazyrouter() {
  if (!this._router) {
    this._router = newRouter(); }}Copy the code

App.listen, app.handle and methods are available in application.js.

Router

Here we find that we have used several Router apis, such as:

router.handle

router.route

route[method]

So let’s look at the Router class. The following code is simplified from the source code:

// router/index.js
var setPrototypeOf = require('setprototypeof');

var proto = module.exports = function () {
  function router(req, res, next) {
    router.handle(req, res, next);
  }

  setPrototypeOf(router, proto);

  return router;
}
Copy the code

This code is strange to me. When we execute the new Router(), we actually execute the new Proto (). The new Proto () is not strange to me, but the way it sets up the prototype. I mentioned earlier in my object-oriented article on JS that if you want to add class methods to a class you can write something like this:

function Class() {}

Class.prototype.method1 = function() {}

var instance = new Class();
Copy the code

This way instance.__proto__ will point to class. prototype and you can use instance.method1.

Setprototypeof (Object. Setprototypeof (obj, prototype)) is a third party library that sets a prototype for an Object. The whole point of setPrototypeof is to be compatible with the old standard JS, which is to add some polyfill, which is the code right here. So:

setPrototypeOf(router, proto);
Copy the code

Router.__proto__ refers to proto, the router is the object you return when new Proto () is executed, and the router gets all the methods on Proto. A method like router.handle can then be mounted to Proto, called Proto.handle.

Add a Class method to the router, but why not use it in a roundabout way, as I did with the Class above? I don’t know, maybe there’s some historical reason.

Routing architecture

Now that we know the basic structure of the Router, to understand the specific code of the Router, we also need to understand the overall routing architecture of Express. Take our two sample apis:

get /api/users

post /api/users

We find that their paths are the same, both/API /users, but their request methods, that is, method, are different. Express extracts the path Layer as a class called Layer. However, for a Layer, we only know its path and cannot determine a route without method. Therefore, an attribute route is added to the Layer, and an array is stored in the route. Each item of the array stores the corresponding method and the callback function Handle. The whole structure you can think of looks like this:

const router = {
  stack: [
    // There are many layers inside
    {
      path: '/api/users'
      route: {
      	stack: [
          // There are multiple methods and callback functions stored in it
          {
            method: 'get'.handle: function1
          },
          {
            method: 'post'.handle: function2
          }
        ]
    	}
    }
  ]
}
Copy the code

Knowing this structure, we can guess that the whole process can be divided into two parts: registering routes and matching routes. When we write the app.get and app.post methods, we add layer and route to the router. When a network request comes, it actually traverses layer and route to find the corresponding handle and execute it.

Pay attention torouteEach item in an array should be stored using a new data structure, for examplerouteItemAnd so on. butExpressI didn’t do that, BUT I combined it withlayerAll together. Here you golayeraddedmethodandhandleProperties. This can be confusing when you first look at the source code becauselayerCoexist inrouterthestackAnd on theroutetheOn the stack“Has two kinds of responsibilities.

router.route

This is one of the methods we called earlier when we registered the route. Recall the previous route registration methods, such as app.get:

app.get = function (path) {
  this.lazyrouter();

  var route = this._router.route(path);
  route.get.apply(route, Array.prototype.slice.call(arguments.1));
  return this;
}
Copy the code

Add layer and route to router when registering route. Router. Route code is easy to write:

proto.route = function route(path) {
  var route = new Route();
  var layer = new Layer(path, route.dispatch.bind(route));     // The arguments are path and the callback function

  layer.route = route;

  this.stack.push(layer);

  return route;
}
Copy the code

Layer and Route constructors

The above code creates Route and Layer instances, and the constructors for these two classes are fairly simple. Just declare and initialize arguments:

// layer.js
module.exports = Layer;

function Layer(path, fn) {
  this.path = path;

  this.handle = fn;
  this.method = ' ';
}
Copy the code

// route.js
module.exports = Route;

function Route() {
  this.stack = [];
  this.methods = {};    // A hash table to speed up lookup
}
Copy the code

route.get

We’ve seen that app.get actually calls route.get with this line of code:

route.get.apply(route, Array.prototype.slice.call(arguments.1));
Copy the code

Route. get adds layer to route.stack.

var methods = ["get"."post"];
methods.forEach(function (method) {
  Route.prototype[method] = function () {
    // Multiple callback functions can be passed in
    var handles = flatten(slice.call(arguments));

    // Create a layer for each callback and add it to the stack
    for (var i = 0; i < handles.length; i++) {
      var handle = handles[i];

      // Each handle should be a function
      if (typeofhandle ! = ="function") {
        var type = toString.call(handle);
        var msg =
          "Route." +
          method +
          "() requires a callback function but got a " +
          type;
        throw new Error(msg);
      }

      // Note that the layer is layer.route.layer
      // The path of the first layer has been compared, so here is the second layer, path can be set to /
      var layer = new Layer("/", handle);
      layer.method = method;
      this.methods[method] = true; // Set methods to true for quick lookups
      this.stack.push(layer); }}; });Copy the code

Router.handle (router) {router.handle ();

router.handle

App. handle is a router that is called by app.handle. The router structure is to add the layer and router to the stack. So all router.handle needs to do is find the corresponding layer and router from router.stack and execute the callback function:

// A function that actually handles routing
proto.handle = function handle(req, res, done) {
  var self = this;
  var idx = 0;
  var stack = self.stack;

  // Next method to find the corresponding layer and callback function
  next();
  function next() {
    // Use the third-party library parseUrl to get the path. If there is no path, return it directly
    var path = parseUrl(req).pathname;
    if (path == null) {
      return done();
    }

    var layer;
    var match;
    var route;

    while(match ! = =true && idx < stack.length) {
      layer = stack[idx++]; Layer = stack[idx]; Perform idx++;
      match = layer.match(path); // Call layer.match to check whether the current path matches
      route = layer.route;

      // If no match is found, exit the loop
      if(match ! = =true) {
        continue;
      }

      // Layer matches, but there is no route
      if(! route) {continue;
      }

      // Check if there is a method on the route
      var method = req.method;
      var has_method = route._handles_method(method);
      // If there is no corresponding method, there is no matching method
      if(! has_method) { match =false;
        continue; }}// If there is no match at the end of the loop, it will be done
    if(match ! = =true) {
      return done();
    }

    // If a match is found, the corresponding callback function is executed
    returnlayer.handle_request(req, res, next); }};Copy the code

The above code also uses several Layer and Route instance methods:

Layer. match(path): checks whether the path of the current layer matches.

Route. _handles_method(method) : Checks whether the method of the current route matches.

Layer. handle_REQUEST (REq, RES, Next) : Use layer’s callback function to handle requests.

These methods don’t seem complicated, so we’ll implement them one by one.

There’s actually a question here. From him the whole matching process, he actually is looking for the router. The stack. This layer, layer but in the end the callback is on the router should be performed. The stack. The layer. The route. The stack. The layer. The handle. How did this through the router. Stack. Find the final layer router. Stack. Layer. The route. The stack. The layer. The handle to perform?

This goes back to our previous router.route method:

proto.route = function route(path) {
  var route = new Route();
  var layer = new Layer(path, route.dispatch.bind(route));

  layer.route = route;

  this.stack.push(layer);

  return route;
}
Copy the code

The new Layer callback is route.dispatch.bind(route), which then goes to route.stack to find the correct Layer to execute. So the real process of router.Handle is:

findpathThe matchinglayer

Come up withlayerOn therouteSee if there’s a matchmethod

layerandmethodIf they all match, call them againroute.dispatchFind the actual callback function to execute.

So there’s another function that needs to be implemented,route.dispatch.

layer.match

Layer. match is a function that checks whether the current path is a match, using a third-party library called path-to-regexp that converts a path to a regular expression for subsequent matches. This library also appears in the react-Router source.

var pathRegexp = require("path-to-regexp");

module.exports = Layer;

function Layer(path, fn) {
  this.path = path;

  this.handle = fn;
  this.method = "";

  // Add a matching re
  this.regexp = pathRegexp(path);
  // Fast match /
  this.regexp.fast_slash = path === "/";
}
Copy the code

Then you can add the match instance method:

Layer.prototype.match = function match(path) {
  var match;

  if(path ! =null) {
    if (this.regexp.fast_slash) {
      return true;
    }

    match = this.regexp.exec(path);
  }

  // If no match is found, return false
  if(! match) {return false;
  }

  // Otherwise return true
  return true;
};
Copy the code

layer.handle_request

Layer. handle_request is a method used to call a specific callback function, which is executed using layer.handle:

Layer.prototype.handle_request = function handle(req, res, next) {
  var fn = this.handle;

  fn(req, res, next);
};
Copy the code

route._handles_method

Route. _handles_method is used to check whether the current route contains the required method, because a methods object was added earlier, which can be used for quick lookup:

Route.prototype._handles_method = function _handles_method(method) {
  var name = method.toLowerCase();

  return Boolean(this.methods[name]);
};
Copy the code

route.dispatch

The route. The dispatch is actually the router. The stack. The layer of the callback function, role is to find the corresponding router. Stack. Layer. The route. The stack. The layer. The handle and executed.

Route.prototype.dispatch = function dispatch(req, res, done) {
  var idx = 0;
  var stack = this.stack; // Note that the stack is route.stack

  // If stack is empty, do it directly
  // 这里的done其实是router.stack.layer的next
  // Execute the next router.stack.layer
  if (stack.length === 0) {
    return done();
  }

  var method = req.method.toLowerCase();

  / / the next method is actually in the router. The stack. The layer. The route. The find method matched layer on the stack
  // Execute layer's callback if it finds it
  next();
  function next() {
    var layer = stack[idx++];
    if(! layer) {return done();
    }

    if(layer.method && layer.method ! == method) {returnnext(); } layer.handle_request(req, res, next); }};Copy the code

Express routing structure, registration and execution process are completed, and the corresponding official source code is posted:

The Router class: github.com/expressjs/e…

The Layer class: github.com/expressjs/e…

The Route class: github.com/expressjs/e…

The middleware

As can be seen from the previous structure, a network request will go to the first layer of the router, and then call next to the second layer. When the path of the layer is matched, the callback will be executed, and then all the layers will be completed in this way. So what is middleware? Middleware is a layer whose path defaults to /, which applies to all requests. With this in mind, the code is simple:

// application.js

// app.use is called router.use
app.use = function use(fn) {
  var path = "/";

  this.lazyrouter();
  var router = this._router;
  router.use(path, fn);
};
Copy the code

Add another layer to router.use:

proto.use = function use(path, fn) {
  var layer = new Layer(path, fn);

  this.stack.push(layer);
};
Copy the code

conclusion

ExpressAlso using native APIShttp.createServerTo achieve.
ExpressThe main job of thehttp.createServerTo build a routing structureRouter.
This routing structure consists of many layerslayerComposition.
A middleware is onelayer.
Routing is another onelayer.layerThere’s one onpathProperty to represent the API path it can process.
pathThere might be different onesmethod, eachmethodThe correspondinglayer.routeOn the onelayer.
layer.routeOn thelayerDespite the name androuterOn thelayerThe same, but the functional focus is different, which is also a point of confusion in the source code.
layer.routeOn thelayerThe main parameters ofmethodandhandleIf themethodIf a match is found, the correspondinghandle.
The whole route matching process is traversalrouter.layerA process of.
Each request comes in and goes through all of themlayerIf a match is made, the callback will be executed, and one request may match more than onelayer.
In general,ExpressThe code doesn’t feel perfect, especiallyLayerClass has two responsibilities, with emphasis on software engineeringSingle responsibilityThe lack of principles also led toRouter.Layer.RouteThe invocation relationship between the three classes is a bit confusing. And the use of inheritance and stereotypes is very old. Maybe it’s the imperfection that’s driving itKoaWe’ll look at that in the next articleKoaSource code bar.
ExpressIn fact, the originalreqandresExtensions have been made to make them easier to use, but this is really just a syntax candy that doesn’t have much impact on the overall architecture, so I won’t cover it in this article.

The code has been uploaded to GitHub, so it is better to play with the code while reading the article:Github.com/dennis-jian…

The resources

Official document of Express: Expressjs.com/

Express official source: github.com/expressjs/e…

At the end of this article, thank you for your precious time to read this article. If this article gives you a little help or inspiration, please do not spare your thumbs up and GitHub stars. Your support is the motivation of the author’s continuous creation.

Welcome to follow my public numberThe big front end of the attackThe first time to obtain high quality original ~

“Front-end Advanced Knowledge” series:Juejin. Cn/post / 684490…

“Front-end advanced knowledge” series article source code GitHub address:Github.com/dennis-jian…