Recently I was writing a scaffold, in which redis is used in many scenarios, so I summarized its common use scenarios

  • Links to this article:
  • Making backup:

The cache

> set User:1:name shanyue EX 100 NX
OK
> get User:1:name
"shanyue"Copy the code

Caching is one of the most visible uses of Redis and can be done using just set/get, but there are a few considerations

  • How to set up the cache better
  • How to keep the cache consistent with upstream data
  • How to solve the cache collapse, cache breakdown problem

Session: user login and verification code

> set 5d27e60e6fb9a07f03576687 '{"id": 10086, role: "ADMIN"}' EX 7200
OK
> get 5d27e60e6fb9a07f03576687
"{\"id\": 10086, role: \"ADMIN\"}"Copy the code

This is also a common scenario, but you can also consider using JWT as opposed to stateful sessions, both with advantages and disadvantages

  • Json Web Token Practical login and verification code verification

The message queue

> lpush UserEmailQueue 1 2 3 4
lpop UserEmailQueue
> rpop UserEmailQueue
1
> rpop UserEmailQueue
2Copy the code

Redis queues can be thought of as distributed queues, with producers stuffing data at one end and consumers putting data at the other end: (Lpush/rPOP, rpush/ LPOP). There are some drawbacks, and these can be fatal, but there are some scenarios where it’s okay to lose a few messages

  1. Without ACK, messages may be lost
  2. You need to doredisPersistent configuration of

Filter (dupefilter)

> sadd UrlSet http://1
(integer) 1
> sadd UrlSet http://2
(integer) 1
> sadd UrlSet http://2
(integer) 0
> smembers UrlSet
1) "http://1"
2) "http://2"Copy the code

As a distributed crawler framework, scrapy-Redis uses the data structure of Redis Set to reprocess urls to be crawled.

# https://github.com/rmax/scrapy-redis/blob/master/src/scrapy_redis/dupefilter.py
def request_seen(self, request):
    """Returns True if request was already seen.
    Parameters
    ----------
    request : scrapy.http.Request
    Returns
    -------
    bool
    """
    fp = self.request_fingerprint(request)
    added = self.server.sadd(self.key, fp)
    return added == 0Copy the code

However, when there are too many urls, there will be a memory problem

A distributed lock

Set Lock:User:10086 06be97fC-f258-4202-b60b-8d5412dd5605 EX 60 NX # If redis. Call ("get",KEYS[1]) == ARGV[1] then return Redis. Call ("del",KEYS[1]) else return 0 endCopy the code

This is one of the simplest standalone distributed locks, with the following points

  • EXIndicates that the lock will expire and be released
  • NXGuaranteed atomicity
  • When unlocking a resource, compare the UUID generated for the resource to avoid a misunderstanding lock

When you are using distributed locks to address performance issues such as distributed timed tasks preventing multiple executions (good idempotent), and since a single point of Redis is less likely to fail, you can use this standalone version of distributed locks.

Rate Limit

Limiting traffic, which allows only a certain number of requests to pass per unit of time, has two key parameters

  • Window, unit time
  • Max, maximum number of requests

The most common scenario is that the SMS verification code can be sent only twice a minute

FUNCTION LIMIT_API_CALL(ip): current = GET(ip) IF current ! = NULL AND current > 10 THEN ERROR "too many requests per second" ELSE value = INCR(ip) IF value == 1 THEN EXPIRE(ip,1) END PERFORM_API_CALL() ENDCopy the code

You can use counters to limit the flow of REQUESTS from the API, but there are several issues to be aware of

  1. There are twice as many requests within the limit of smooth sliding window time
  2. Race Condition

This can be further restricted programmatically based on the TTL key, or filtered in real time using a LIST that maintains a timestamp for each request call. Here is a Rate Limter implemented by Node. Node-rate-limiter-flexible

this.client .multi() .set(rlKey, 0, 'EX', secDuration, 'NX') .incrby(rlKey, points) .pttl(rlKey) .exec((err, res) => { if (err) { return reject(err); } return resolve(res); }) if (res.consumedPoints > this.points) { // ... } else if (this.execEvenly && res.msBeforeNext > 0 && ! res.isFirstInDuration) { // ... setTimeout(resolve, delay, res); } else { resolve(res); }Copy the code

  • node-rate-limiter-flexible
  • Mail sending, flow limiting, leak bucket and token bucket

Distributed websocket

You can communicate between Websocket servers via Redis PUB/SUB. Refer to the following items

  • socket.io-redis

I am Shanyue94. Please add shanyue94 on wechat to communicate with me. In addition, you can pay attention to my public account [The road to full stack Growth].