An overview of the
In the article of Redis, I wrote an article about the three major problems of Redis cache before. The total number of readers is nearly 800, and it is quite difficult to reach this number of readers with only about 3k fans.
[] [] [] [] [] [] [] []
The three cache problems are only a small part of Redis knowledge, if you want to study Redis in depth, there are more knowledge points to learn.
So today brings us to a common interview question: what if your Redis memory is full? Use Redis as a cache for a long time, one day it will be full, right?
The maxMemory configuration parameter in Redis can set the size of Redis memory.
Configuration file in Redisredis.confFile, configurationmaxmemoryThe size parameters of:
100mbDon’t get me wrong, I’m just giving you a sense of the size of this parameter, which small companies usually set to3GLeft and right.
In addition to making the configuration take effect in the configuration file, you can also configure the parameters in the form of command lines. The specific command lines are as follows:
127.0.0.1:6379> config get maxMemory 127.0.0.1:6379> config set maxMemory to 100MB 127.0.0.1:6379> config set maxMemory 100mbCopy the codeIf the actual storage exceeds the size of the Redis configuration parameter, Redis has a elimination strategy. The key that needs to be eliminated is eliminated and a clean piece of memory is sorted out for the use of new key values.
Next, we will talk about the elimination strategy in Redis in detail, and deeply understand the principle and application scenarios of each elimination strategy.
Elimination strategy
Redis offers 6 types of elimination strategies, of which the default is Noeviction, the 6 elimination strategies are designed as follows:
noeviction(The default policy) : If the memory size reaches the threshold, all memory request instructions will report an error.allkeys-lru: All keys are usedLRU algorithmElimination.volatile-lruAll:The LRU algorithm is used for keys with an expiration timeElimination.allkeys-random: All keys are usedRandom selectionIn the way of elimination.volatile-randomAll:Keys with expiration time are randomly eliminatedIn the way of elimination.volatile-ttl: All keys with expiration time setAccording to the expiration time, the earlier the expiration, the faster it will be eliminated.
Allkeys-lru can be used if some of the data in Redis is hot and the rest is unpopular, or if we are not sure about the cache access distribution of our application.
If all data are accessed at roughly the same frequency, the AllKeys-random elimination strategy can be used.
If you want to configure a specific elimination strategy, you canredis.confIn the configuration file, the specific configuration is as follows:
127.0.0.1:6379> config get maxmemory-policy: allkeys-lru 127.0.0.1:6379> config set maxmemory-policy allkeys-lruCopy the codeIn the introduction of 6 kinds of elimination strategy, said the LRU algorithm, so what is the LRU algorithm?
LRU algorithm
LRU(Least Recently Used) indicates the Least Recently Used key, that is, the key that has been accessed Least in the recent time. The algorithm filters out data based on the historical access records of data.
The core idea is that if a key is rarely used in the recent past, it will rarely be accessed in the future.
In fact, the LRU implemented by Redis is not a real LRU algorithm, that is, nominally we use the LRU algorithm to eliminate the key, but in fact, the eliminated key is not necessarily the real most useless.
Redis uses an approximate LRU algorithm to eliminate keys through random collection. Five keys will be randomly selected each time, and then the least recently used key will be eliminated.
The five keys are only the default number. The specific number can also be configured in the configuration file, as shown in the following figure:
In order to implement the LRU algorithm based on time, Redis must add an additional memory space for each key to store the time of each key, which is 3 bytes.
The approximate LRU algorithm was optimized in Redis 3.0 to maintain memory for a candidate pool of size 16.
When sampling data is randomly selected for the first time, the data will be put into the candidate pool, and the candidate pool data will be sorted according to time.
After the second selection, only data less than the minimum time in the candidate pool will be put into the candidate pool.
When the candidate pool is full at some point, the oldest key is pushed out of the candidate pool. The key with the shortest recent access time is selected from the candidate pool to be eliminated.
The purpose of this is to select the nearest key value that seems to be the least recently accessed, so that the key value can be eliminated correctly, because the minimum time in the randomly selected sample may not be the minimum time in the real sense.
However, the LRU algorithm has a disadvantage: if a key value has not been accessed before, but was recently accessed, it will be considered as hot data and will not be eliminated.
However, some data is often accessed in the past, but not in the recent time, which leads to these data is likely to be eliminated, so that hot data will be misjudged and eliminated.
So in Redis 4.0, in addition to the LRU algorithm, a new LFU algorithm is added, so what is the LFU algorithm algorithm?
LFU algorithm
The LFU(Least Frequently Used) is the key that is Frequently Used in the recent time range. It is determined by the frequency of the access times in the recent time range.
Its core idea is: according to the recent access frequency of the key to eliminate, less access key priority elimination, vice priority retention.
The LFU algorithm reflects the heat of a key, and is not considered as hot data because the LRU algorithm is accessed occasionally.
The LFU algorithm supports volatile- LFU and AllKeys-LFU.
These are Redis’ six weeding strategies, designed to tell us what to do, but when? Without saying that, let’s take a closer look at when Redis will implement the elimination strategy.
Delete an expired key policy
There are three delete operations in Redis.
- Scheduled deletion: A timer is created to periodically delete keys.
- Lazy deletion: The system checks the expiration time of the key only when the key is accessed again. If the key has expired, the system deletes it.
- Periodically delete: Delete expired keys every once in a while.
Timed deletes are memory friendly, timed to clean up space, but not CPU friendly. Programs need to maintain a timer, which consumes CPU resources.
Lazy deletes are CPU-friendly because the CPU does not need to maintain any additional operations, but they are not memory friendly because some keys are never accessed and will always use up memory.
Periodic deletion is a compromise between the above two solutions. The expired keys are deleted every once in a while. That is, the key is deleted at a reasonable time according to specific services.
The key is deleted by controlling the deletion interval to reduce CPU resource consumption and rationalize the deletion operation.
Elimination of RDB and AOF
There are two methods of persistence in Redis, RDB and AOF. For detailed introduction of these two methods of persistence, please refer to this article [].
In RDB, a snapshot is used to obtain a copy of the data in memory at a certain point in time. When creating an RDB file, you can run the save and BGsave commands to create an RDB file.
Neither of these commands will save the expired key to the RDB file, which can also remove the expired key.
When starting Redis to load RDB files, the Master will not load expired keys, while the Slave will load expired keys.
In AOF mode, Redis provides Rewite optimization measures by executing REWRITEAOF and BGREWRITEAOF commands respectively. Neither of these two commands will write expired keys into AOF files and delete expired keys.