concept

Redis is an in-memory database, data is stored in memory, once the server process exits, the data is lost, so Redis needs to find a way to persist the data stored in memory to disk.

Redis provides two types of persistence:

  1. Redis Database (RDB) : generates an RDB file in key-value format.
  2. AOF (Append Only File) : saves write commands during Redis execution.

generate

The generation of RDB

  1. The SAVE command blocks the Redis server process until the RDB file is created, and the server cannot process any command requests while the server process is blocked.

  2. The BGSAVE command gives birth to a child process, which then creates the RDB file, and the server process (parent) continues to process the command requests.

If the change of the two keys is transactional, you need to add the transaction manually. Otherwise, the two keys may be inconsistent during backup.

In addition to executing the command actively, we can set multiple save conditions with the save option, and the server will execute the BGSAVE command if any of the conditions are met:

save 900 1
save 300 10
save 60 10000
Copy the code

The BGSAVE command is executed if any of the following three conditions are met:

  1. The server made at least one change to the database within 900 seconds.
  2. The server made at least 10 changes to the database within 300 seconds.
  3. The server made at least 10,000 changes to the database within 60 seconds.

The generation of AOF

When AOF persistence is enabled, the server appends a write command to the end of the aOF_buf buffer of the server state in a protocol format.

In modern operating systems, when a user writes a file, the operating system usually stores the written data temporarily in a memory buffer until the buffer is full or the specified time limit is exceeded before the data is actually written to disk. This can cause data in the buffer to not be written to disk and the computer to stop, resulting in data loss.

The value of the appendfsync option determines the efficiency and security of AOF persistence:

  1. alwaysSynchronization to disk every time, low efficiency, high security;
  2. everysecSynchronization to the disk every second, the efficiency is fast enough, high security, only one second of data loss;
  3. noSynchronization to disks is controlled by the operating system, which is the fastest and has the lowest security.

AOF the rewrite

Because AOF stores write commands, as the server runs, the more times the same key is operated on, the more write commands will be generated for a single key, and the larger the AOF file will be, the longer the restore time will be.

To solve the problem of bloated AOF, Redis provides AOF file rewrite.

AOF rewrite does not compress old AOF files. Redis will read data from the database, generate corresponding write commands, and write to the new AOF file. When the new AOF file overwrites all data write commands, the old AOF file can be replaced.

AOF rewriting can be carried out in the background, the new data generated in the rewriting process will be written into the AOF rewriting buffer, and when the rewriting is over, the write command of the buffer will be appended to the new AOF file.


load

RDB load

The loading of RDB files is done automatically when the server starts, so Redis has no special commands for loading.

Since AOF files are usually updated more frequently than RDB files:

  1. If AOF persistence is enabled on the server, AOF files are used to restore database state in preference.
  2. The RDB file is used by the server to restore the database state only when AOF persistence is turned off.

AOF load

AOF contains all the write commands. The server can restore the state before the server is shut down by reading and re-executing the write commands saved in the AOF file.

Redis reads the AOF file and restores the database state:

  1. Create a pseudo-client without network connection: because the commands of Redis can only be executed in the client context, and the commands used to load AOF files come directly from the AOF file rather than the network connection, the pseudo-client used to restore data does not need a network connection;
  2. Analyze and read a write command from AOF file;
  3. Use pseudo client to execute read write command;
  4. Repeat steps 2 and 3 until all commands in the AOF file have been executed.

data

Redis Design and Implementation – Huang Jianhong – wechat Reading [1]

This post was originally posted on my personal blog https://chaohang.top

The author is Zhang Xiaochao

Please indicate the source of reprint

Please follow my wechat official account [Chaochao Won’t fly] to get the latest updates.


The resources

[1]

Design and implementation of Redis – Huang Jianhong – WeChat reading: https://weread.qq.com/web/reader/d35323e0597db0d35bd957bk7cb321502467cbbc409e62d

This article is formatted using MDNICE