Editor’s note: As the saying goes, “If there is not enough concurrency, the machine will come”, adding machines is the easiest and most expensive thing to do when we face high concurrency requests. But what can we do with limited resources other than optimize the code? Today, we have invited @Youma to share his experience in this area with us, hoping to help you.

— –

It’s not enough to consider security in order to develop solid Web apis; there’s also a strategy to deal with mass access. Not only Web API services, but any service that is publicly available on the Web will occasionally encounter mass access from outside, such as real-time hot spots like “Lu Han guan Xiaotong Announces Relationship”. When a server encounters a large number of visits, it runs out of resources to handle them and cannot provide services. Not only are these massive accesses, but no one can connect to the server.

We can easily access the Web API through programs, so the API server is more likely to encounter high access load. For this problem, just like ordinary Web applications, we can expand the API service, which is the right approach, but we will not discuss the expansion solution in this article. Next, I’ll discuss some of the important aspects of speed limits when dealing with mass access, and what to do in a ThinkJS development project.

Restrict user access

To solve the problem of sudden mass access, the most realistic way is to limit the number of accesses per user. This is to determine the maximum number of access times for a user. If the number of access times exceeds the maximum, the server rejects the request and returns an error message when the user attempts to access the system again. For example, if a user is allowed to invoke the interface for obtaining the SMS verification code only 20 times within 10 minutes, the server returns an error message when the user initiates the 21st request within 10 minutes, and the access is resumed 10 minutes later. If the access speed limit is implemented, the following three problems should be solved first:

  • How to determine the speed limit
  • How to determine the unit of time limit
  • When is the speed limit reset

Determine the speed limit

For frequently updated query apis, users need frequent access to the latest data. If you set it to 10 times per hour, users will not be satisfied and will switch to alternative services. The purpose of limiting access rates is to prevent the server from being overwhelmed by massive access in a short period of time. However, it is not worth the loss if the user is made to use it inconveniently. Therefore, it is important to know as much as possible under what conditions the API is being used, and then determine the rate limit.

Determine the speed limit time unit

According to the online service is different, some will be in one day as a time unit, visited but it is a bit long for many apis, assume that the user is writing scripts access API, begin to access number of time units is not clear, it may need to let him wait 24 hours to continue to access the API, or change a account. If we use 10 minutes as the unit of access times, if we exceed the access limit, we only need to wait 10 minutes to continue access. Although the unit time setting is closely related to the data returned by the API, most publicly available apis set the unit time to about an hour.

Determines the time to reset the speed limit value

How does the server return a response message when the user exceeds the access threshold? In this case, the 429 Too Many Request status code prepared in the HTTP protocol can be returned. 429 Status code Is defined in RFC 6585 released in April 2012. When a user initiates too many requests within a certain period of time, the server can return this status code to indicate an error. The RFC document describes the status code as follows:

429 Too Many Requests

   The 429 status code indicates that the user has sent too many
   requests in a given amount of time ("rate limiting").

   The response representations SHOULD include details explaining the
   condition, and MAY include a Retry-After header indicating how long
Copy the code

As you can see from the above description, the response message should contain details of the error and be retry-after to tell the user how long it will take to access the API. Retry-after the header indicates how long the client will have to wait before it can be accessed again. This header is marked with MAY in the RFC document, indicating that there is no problem with not sending the header, but that it is more friendly to include the header in the response body.

In addition, retry-After is not a response header dedicated to the 429 status code. This header is defined in RFC 7231 for HTTP 1.1 and is also included in the response body with the 503 and 3XX series. The retry-after header specifies the time in seconds and can also use date details as described in the RFC documentation:

Retry-After

   Servers send the "Retry-After" header field to indicate how long the
   user agent ought to wait before making a follow-up request.  When
   sent with a 503 (Service Unavailable) response, Retry-After indicates
   how long the service is expected to be unavailable to the client.
   When sent with any 3xx (Redirection) response, Retry-After indicates
   the minimum time that the user agent is asked to wait before issuing
   the redirected request.

   The value of this field can be either an HTTP-date or a number of
   seconds to delay after the response is received.

     Retry-After = HTTP-date / delay-seconds

   A delay-seconds value is a non-negative decimal integer, representing
   time in seconds.

     delay-seconds  = 1*DIGIT
Copy the code

Speed limiting information is passed through HTTP responses

It is very friendly to inform users of the current access limit, the number of access counts that have been used, and when to reset the access rate limiting. If this information is not returned, the user may attempt to access the interface API multiple times to determine whether the speed limit is lifted, which will undoubtedly increase the strain on the server.

Speed limiting information can be placed in the header of the response message, or as part of the body data of the response message. It is now the de facto standard to place speed limiting information in the header of the response message.

The first name instructions type
X-RateLimit-Limit Access upper limit per unit time Integer
X-RateLimit-Remaining Remaining number of visits Integer
X-RateLimit-Reset Access count Resets the time UTC epoch seconds

Take a look at GitHub’s speed limit policy. GitHub uses the above three response headers without the retry-after headers. Authenticated requests can be accessed 5000 times per hour, while unauthenticated requests can be accessed 60 times per hour.

However, the time window of Twitter speed limit policy is 15 minutes, which is much smaller than GitHub’s time window. Because Twitter data updates relatively quickly, the time window setting is smaller to meet users’ requirements for obtaining the latest data. Twitter uses response headers similar to the above three to send the speed limit information x-rate-limit-limit,x-rate-limit-remaining, and x-rate-limit-reset. There are two initial schemes for GET requests, 15 requests in 15 minutes and 180 requests in 15 minutes, and only authentication access is allowed.

By comparing the speed limit policies of GitHub and Twitter, we can know that as long as the speed limit information is accurately conveyed, the response header can be defined by itself. The key point is that the semantics are clear and it cannot conflict with other standard headers.

Implement API speed limit control in ThinkJS

To achieve API access rate limiting, the number of times that each user and application accesses the API needs to be counted, which is generally recorded by storing key-value pairs such as Redis. ThinkJS implements the think-Ratelimiter middleware to limit the speed of actions in combination with its own route mapping. You need to configure the following in the middlemandle.js to implement a simple speed limiting policy.

// in middleware.js
const redis = require('redis');
const { port, host, password } = think.config('redis');
const db = redis.createClient(port, host, { password });
const ratelimiter = require('think-ratelimiter');

module.exports = {
  // after router middleware
  {
    handle: ratelimiter,
    options: {
      db,
      errorMessage: 'Sometimes You Just Have To Slow Down'.headers: {
        remaining: 'X-RateLimit-Remaining'.reset: 'X-RateLimit-Reset'.total: 'X-RateLimit-Limit'
      },
      resources: {
        'test/test': { // Key is the concatenation of controller/action
          id: ctx= > ctx.ip,
          max: 5.duration: 7000 // ms}}}},}Copy the code

The response body header x-ratelimit-reset indicates the time at which access can be resumed, along with the retry-after header, which is the number of seconds from the recovery time.

conclusion

In Web applications developed by ThinkJS, you can use middleware and add configuration to achieve simple speed limiting. If you provide Web API services with high traffic or require paid access, you need to add a layer to do speed limiting related things before the real logic. You can implement a services/ratelimit.js in ThinkJS, and then implement logic like speed limits in your project’s Base Controller.

References:

  • Web API Design and Development
  • RFC 6585
  • RFC 7231
  • Retry-After