“This is the fourth day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

What is caching? Why cache?

Common memory caches are Redis and Memcached

1. Use caching to relieve the pressure of concurrent access to relational databases (hot data)

2, reduce response time: memory I/O speed is faster than disk

3. Improve throughput. Redis and other in-memory databases can support large concurrency on a single machine

Redis common data types and usage scenarios

1. String – Used for simple KV key-value pair storage, such as counters

2, List – to achieve a two-way linked List, such as user concerns, fan List

3. Hash – Key-value pairs used to store information related to each other

4. Set – Stores non-repeating elements, such as user concerns

5. Sorted Set – Real-time information ranking

The built-in implementation of Redis

1. String uses integers or SDS

2. Use ziplist or double Linked List

3. Hash uses ziplist or Hashtable

Set uses intSet or hashtable

SortedSet uses skiplist skip lists

Refer to Redis design and implementation

What is a skip list?

What are the persistence methods of Redis

1, RDB,

Put the data snapshot in the disk binary, dump.rdb

Snapshots are implemented by saving Redis database state to a compressed binary file at specified intervals

The advantage is that you can make a snapshot to the database for a period of time to record the status of a certain time

The disadvantage is that if you go down between two intervals, you lose data in that interval

2, AOF

append only file

Each write command is appended to appendone.aof

The advantage is that there is no data loss

The disadvantage is that THE AOF will be relatively large and the recovery speed is slow

Modifying the Configuration

The affairs of Redis

1. Redis transactions package multiple requests and execute multiple commands in sequence at one time

2. Redis executes transactions through MULTI, EXEC, WATCH and other commands

3. Use the following command in Python to implement transactions

python redis-py pipline = conn.pipline(transaction=True)
Copy the code

Redis implements distributed locking

Use cached patterns

Data consistency between cache and database

The cache to penetrate

Because a large number of cache can not find the database to take, the database does not want the data, this time the database pressure is very big

Many mindless crawlers traverse article ids in an incremental fashion, which causes a lot of stress on the site being crawled

The site can also cache data that is returned as None if not found

Later, when relevant data is available, the corresponding cache is removed or a shorter timeout is set

Cache breakdown

Cache breakdown is caused by a large number of requests to the database due to key failure of a large number of hot spots

Solution:

Distributed lock: the thread that acquires the lock pulls data from the database to update the cache, and the other threads wait for the cache to update and retrieve it from the cache

Asynchronous background update: Background tasks update expired keys

Avalanche problem

The cache is unavailable or a large number of cache keys are invalid at the same time and a large number of requests are directed to the database

1. Multi-level cache: Set different timeout periods for different levels of keys

2, set random timeout: set the key timeout time randomly, do not set all the same, resulting in all invalid

3. Improve system availability and improve monitoring and alarm

To consider