The reason why Redis is called an in-memory database rather than just an in-memory cache is that it has its own persistence mechanism, which can persist the database to disk, and can be recovered in time after downtime. There are two main persistence methods, AOF and RDB. The following is a summary of these two methods.
AOF
The basic concept
The AOF mode, also called Append Only File, writes the corresponding log to the log File after the command is executed. Since the operation is executed before the log is written, the current write operation is not blocked.
When logs are flushed determines how much data Redis loses when it recovers from an outage, and Redis offers three strategies
- Always: write back synchronously. After the command is executed, write back to the disk immediately
- Everysec: Writes back every second, writing cached page data back to disk every second
- No: write back is controlled by the operating system. The operating system determines the flush time.
For the above three policies, the higher the requirement of data reliability, the greater the impact on performance, so how to configure it is a trade-off.
The format of AOF log writing is the contents of the Redis protocol used to execute commands. Redis protocol is a text protocol, which naturally occupies a large space. As it records the operation flow, the constant modification of the same Key makes a lot of historical values recorded in THE AOF log. Redis has introduced AOF rewriting mechanism. How to rewrite and whether it will affect the external services provided by Redis are what we care about. The following is a summary of the AOF rewriting process in the form of questions.
AOF rewrite
1. When to start rewriting
This is determined by the configuration in Redis, which can be set in terms of percentage and file size. The relevant configuration is auto-aof-rewrite-*
2. Will the rewriting process affect Redis’ external services?
No, the overwrite process is done by the backstage child process bgrewriteaof, which blocks because the operating system copies the page tables and other information during the fork process, but after the copy is complete, the child process shares the memory space with the parent process. The parent process also changes memory without affecting the child process due to the Copy On Write mechanism. The child process overwrites the AOF, and the parent process provides services and works at the same time. Overall, it is not affected.
3. Will the rewrite process directly delete the original AOF log?
No, the original log will still be used until the AOF rewrite is complete.
4. The memory space of the rewritten AOF log process is a snapshot, what about the data during the rewritten process?
During the rewrite process, data is written to two caches, one is the original AOF cache and the other is the AOF rewrite cache. When the rewrite is complete, the latest operation recorded in the rewrite buffer is also written to the new AOF file to keep the database up to date, and is replaced after completion
RDB
When you restore Data, you need to run commands from the beginning to the end to restore the Data to the latest state. However, if you can directly take a snapshot of the memory, it will be much faster. This is called RDB, which is called Redis Data Base
- save
- bgsave
Bgsave creates a child process that is responsible for generating the snapshot. How the child process does this is explained in AOF above.
For RDB, another problem is the frequency of snapshots. From the perspective of reliability, you want to create snapshots all the time, but because you copy the entire Redis memory data, the overhead and space are relatively large, and the pressure on the disk is great. At this point, the advantages of AOF are presented, so the mixed mode of AOF and RDB proposed in Redis4.0 is introduced
AOF and RDB mixed mode
Mixed mode uses both methods to generate snapshots: RDB is generated in the specified period of time in the configuration file, and write operations occur in the process to the snapshot file in the form of AOF.