Hi, I’m Shanyue, and this is my new pit: minimal implementation of handwritten source code, with comments for every line of code.

When we are studying a framework or library in depth, source code research is the best way to understand its thinking and design, as well as to better use and avoid unintended bugs.

This is especially true for extremely simple but widely used frameworks/libraries such as KOA and VDOM. To verify that you know enough about it, you can implement a mini-library with only the core functions. As the saying goes, although the sparrow is small, it has all five organs.

For the source code, I will annotate every line of code I can and explain the principles and implementation in an article. The list of current implementations is as follows:

  • mini-koa
  • mini-http-router
  • mini-express
  • mini-webpack (WIP)
  • mini-vdom (WIP)

As ESM has been well supported by browsers, ESM is used to write client-side source code, such as vDOM and React. Use CommonJS for server writing, such as KOA, Express, etc.

How to implement a minimal version of Express

Warehouse: mini – express

Hello, I’m Shanyue.

Express is the most downloaded server-side framework in Node, although it is largely due to webPack’s dependency. Write a mini version of Express today to get a look inside.

Mountain month code implementation

Code is placed in shfshanyue/mini-code:code/ Express

Can read the source code directly, basically every line has comments.

Use NPM run Example or Node example to run the sample code

$ npm run example
Copy the code

Personal opinion about Express

  1. Middleware design for reroute. inexpressAll middleware will passpath-to-regexpTo match the route regex, resulting in some (limited) performance degradation.
  2. querystringDefault middleware. In Express, qs is parsed by built-in middleware on each request, resulting in a certain performance degradation (on-demand parsing in KOA).
  3. Design without Context. Express stores data inreqOf course, it can also be customizedreq.contextUsed to store data.
  4. res.sendThrow data back directly, nonectx.bodyFlexible.
  5. The source code is difficult to understand, and the syntax is old, no KOA code is clear.
  6. expressMany middleware are integrated by default, such as Static.

Middleware design for Express

Middleware in Express can be divided between application level middleware and route level middleware.

// Application level middleware A, B
app.use('/api'.(req, res, next) = > {
    // Application middleware A
    console.log('Application Level Middleware: A')},(req, res, next) = > {
    // Application middleware B
    console.log('Application Level Middleware: B')})// Use app.get to register an application-level middleware (routing), which consists of routing level middleware C and D
app.get('/api'.(req, res, next) = > {
    // Routing middleware C
    console.log('Route Level Middleware: C')},(req, res, next) = > {
    // Routing middleware D
    console.log('Route Level Middleware: D')})Copy the code

In Express, the data structure Layer is used to maintain the middleware, while stack is used to maintain the list of middleware.

All middleware is mounted under router.prototype. stack or route.prototype. stack with the following data structure.

  • App.router. Stack: All application-level middleware (i.eapp.useRegistered middleware).
  • App.router.stack [0].route.stack: all routing level middleware of an application-level middleware (i.eapp.getRegistered middleware).

The following is a pseudo-code data structure for express middleware:

const app = {
  stack: [
    Layer({
      path: '/api'.handleRequest: 'A's middleware handler function '
    }),
    Layer({
      path: '/api'.handleRequest: 'B's middleware handler function '
    }),
    Layer({
      path: '/api'.handleRequest: 'Dispatch: to execute all routing level middleware under this middleware'.// Middleware registered with app.get (application-level routing middleware) will carry the Route attribute to store all routing level middleware of that middleware
      route: Route({
        path: '/api'.stack: [
          Layer({
            path: '/'.handleRequest: 'C middleware handler function '
          }),
          Layer({
            path: '/'.handleRequest: 'D's middleware handler function '})]})})]}Copy the code

Based on the pseudocode above, let’s comb through the process of matching middleware in Express:

  1. Register application level middleware, configure handleRquest and PATH, and configure it according topathgenerateregexp, such as/api/users/:idgenerate/^\/api\/users(? : \ / ([^ # \ \ /?] +? ) / # \ \ /? ? $/i
  2. When the request comes in, the middleware array is traversed according to the middleware’sregexpMatch the request path to get the first middleware
  3. If there is next in the first middleware, go back to step 2 and find the next middleware
  4. Traverse the end

The realization of the Application

Only one function needs to be implemented in the Application layer

  • Abstract and encapsulate HTTP handleRequest
class Application {
  constructor () {
    this._router = new Router()
  }

  Listen handles requests and listens for port numbers, as koA does, or almost all server-side frameworks dolisten (... args) {// Create a service, this.handle is the entry function, in the source code, express app itself is the entry function
    const server = http.createServer(this.handle.bind(this)) server.listen(... args) } handle (req, res) {const router = this._router
    router.handle(req, res)
  }

  // Register application-level middleware, collect all application-level middleware into this._router.stack, and implement the Onion modeluse (path, ... fns) { }// Handle HTTP verbs, such as get, post,
  // Register anonymous application level middlewareget (path, ... fns) { } }Copy the code

Middleware abstraction: Layer

  1. Middleware abstraction
  2. Middleware matching

The middleware performs several functions:

  1. How to determine a match
  2. How to handle requests

The following data structures are designed based on this

Layer({
  path,
  re,
  handle,
  options
})
Copy the code

The re is used to match the request path and is generated according to path. So how do I get the parameters defined in the path? Use capture groups.

At this point, the artifact path-to-regexp is sacrificed, and the path becomes regular. It is favored by both server-side frameworks like Express and Koa and the routing parts of client frameworks like React and Vue.

const { pathToRegexp } = require('path-to-regexp')

pathToRegexp('/')
/ / = > / ^ \ [# \ \ /?] ? $/i

// Can be used to match prefix routes
p.pathToRegexp('/', [] and {end: false })
/ / = > / ^ \ / (? : / # \ \ /? (? = []) | $)? /i

// For parameters, capture the parameters by capturing groups
pathToRegexp('/api/users/:id')
//=> /^\/api\/users(? : \ / ([^ # \ \ /?] +? ) / # \ \ /? ? $/i
Copy the code

With the re, the logic for matching middleware comes naturally, and the code is as follows


// A layer of abstraction for middleware
class Layer {
  //
  Use ('/users/:id', () => {})
  // path: /users/:id
  // handle: () => {}
  constructor (path, handle, options) {
    this.path = path
    this.handle = handle
    this.options = options
    this.keys = []
    // Generate regular expressions according to path
    this.re = pathToRegexp(path, this.keys, options)
  }

  // Check whether the request path matches the middleware, and if so, return the parmas that match
  match (url) {
    const matchRoute = regexpToFunction(this.re, this.keys, { decode: decodeURIComponent })
    return matchRoute(url)
  }
}
Copy the code

Middleware collection

App. use and app.get are used to collect middleware, with relatively simple codes as follows:

class Application {
  // Register application-level middleware, collect all application-level middleware into this._router.stack, and implement the Onion modeluse (path, ... fns) {this._router.use(path, ... fns) }// Handle HTTP verbs, such as get, post,
  // Register anonymous application level middlewareget (path, ... fns) {const route = this._router.route(path)
    // All route-level middleware involved in this application-level middleware is handled in route.prototype.getroute.get(... fns) } }class Router {
  constructor () {
    // Collect all application-level middleware
    this.stack = []
  }

  // Implementation of application level middleware Onion model
  handle (req, res) {
  }

  // 
  // app.use('/users/', fn1, fn2, fn3)
  // The path can be omitted in express, which defaults to all paths. For better understanding of the source code, it is omitted hereuse (path, ... fns) {for (const fn of fns) {
      const layer = new Layer(path, fn)
      this.stack.push(layer)
    }
  }

  // Register application level routing middleware, which is an anonymous middleware, to maintain a set of routing level middleware related to the path,
  route (path) {
    const route = new Route(path)
    // The anonymous middleware's handleRequest function cascades all the routing middleware under the application-level intermediate mount
    // For routing level middleware, full match, i.e. / API will only match/API
    const layer = new Layer(path, route.dispatch.bind(route), { end: true }) 
    layer.route = route
    this.stack.push(layer)
    return route
  }
}
Copy the code

Among them, route.prototype. stack is specially responsible for the collection of route-level middleware, and multiple route-level middleware consists of dispatch function to form an application middleware, among which is an Onion model, which will be discussed in the following.

Middleware and the Onion model

The Onion model is also relatively simple to implement, using next to connect all matching middleware and execute on demand.

function handle (req, res) {
  const stack = this.stack
  let index = 0

  // Invoke the next application level middleware
  const next = () = > {
    let layer
    let match

    while(! match && index <this.stack.length) {
      layer = stack[index++]
      // Check whether the request path matches the middleware, and if so, return the parmas that match
      match = layer.match(req.url)
    }
    // Traverse the middleware, if no path matches, then the status code is 404
    if(! match) { res.status =404
      res.end('NOT FOUND SHANYUE')
      return
    }
    req.params = match.params
    // The function that handles the middleware, and moves down to the next middleware if next() is called in the middleware
    layer.handle(req, res, next)
  }
  next()
}
Copy the code

In comparison, the implementation of routing level middleware onion model is much simpler

function dispatch (req, res, done) {
  let index = 0
  const stack = this.stack
  const next = () = > {
    const layer = stack[index++]

    // If the last one
    if(! layer) { done() } layer.handle(req, res, next) } next() }Copy the code

conclusion

To the end.

Wait a minute. Remember to have breakfast.