Redis provides two persistence mechanisms, RDB and AOF

Redis Database (RDB): the RDB stores snapshot data generated before a certain point in time.

AOF(appends-only File): all command line records are stored as AOF files in the redis command request protocol format

Mixed persistence (after version 4.0) : When AOF is overwritten, the child process saves the snapshot of the current point in time as an RDB file, and then saves the accumulated commands of the parent process as an AOF file.

RDB snapshots can be triggered in two ways

1: indicates that the configuration parameters are passed.

If the number of commands executed in a certain period of time exceeds the threshold, the snapshot is generated immediately

Save 900 1 //900 seconds Save 300 10 // 10 updates in 30 seconds Save 60 10000 // 10000 updates in 60 secondsCopy the code

2: Manually execute bgSave/Save, manually trigger snapshot generation directly execute save will block the main process, bgSave will fork a child process to complete the snapshot

But Redis has a few issues to consider when RDB persistence occurs

1. Check whether Redis stops external services during the RDB snapshot

2. If the service is not stopped, how to process new requests

Let’s move on to Redis

The details of RDB persistence

1: The main process forks a child process

2: The child process shares some of the data space of the main process and puts the shared data into the read-only state. In this process, the child process persists using the RDB protocol

3: In the process of persistence, it is inevitable to have new data written, because part of our data is shared, and two processes have a piece of data at the same time, which will definitely lead to data inconsistency. However, depending on the fork mechanism of the operating system, the data of some memory pages must be modified when modifying. At this time, the copyonwrite operation of the corresponding memory page will be triggered, which will not affect the child process after persistence. After persistence, the main process will recycle the child process

RDB file format

  • Redis RDB files are a very compact format

  • REDIS is a fixed format. If a persistent file does not start with REDIS, an error will be reported

  • 0006 Yes RDB_VERSION Indicates the current RDB version

  • AUX_FIELD_KEY_VALUE_PAIRS are auxiliary fields

  • Data is the saved data. Redis_db is selected first. After db is selected, the data of the key-value pair is divided into the data whose expiration time has been set and the data whose expiration time has not been set

Advantages of RDB persistence

1: Binary data is very compact and fast in data recovery

2: In the process of persistence, the performance is maximized. Fork sub-process to complete write operations, let the main process continue to process commands, and use a separate sub-process for persistence, which ensures the high performance of Redis

Disadvantages of RDB persistence

1: Low data security. RDB is persisted at an interval. If redis fails during the persistence, data will be lost

2: After Linux fork, the kernel sets the permissions of all memory pages in the parent process to readonly, and the address space of the child process points to the parent process. When both parent and child processes read only memory, nothing happens. When one of these processes writes to memory, the CPU hardware detects that the memory page is read-only and triggers a page-fault, falling into an interrupt routine of Kernal. In the interrupt routine, the kernel’s copyonwrite mechanism makes a copy of the triggered exception page, so that the parent process keeps a separate copy. If there are a lot of writes at this time, there will be a lot of paging errors (page exceptions interrupt page-faults), which will cost a lot of performance in replication.

AOF persists the execution process

Enabled with appendOnly Yes

Redis uses single-thread response commands. If each AOF file command is appended to the disk, the processing performance will be greatly affected. Therefore, Redis first writes to the AOF buffer and then writes to the AOF file according to the synchronization policy configured by the user

Appendfsync always # writes data to disk manually after each update operation. Appendfsync everysec # synchronizes once per second. Appendfsync no # indicates that the operating system performs data cache synchronization to disk (fast response client does not perform data synchronization to AOF, file synchronization is the responsibility of the operating system, usually takes up to 30 seconds).Copy the code

AOF rewrite mechanism

As commands are written to AOF, files become larger and larger. To solve this problem, Redis introduced AOF rewriting to reduce file size. AOF file rewriting is the process of converting data in the Redis process into write commands to synchronize to a new AOF file. The AOF file rewriting mechanism can be triggered manually or automatically

Manual trigger: the bgreweuteaof command

Automatic trigger:

Auto-aof -rewrite-percentage 100 # indicates the ratio of the current aof file space to the aof file space after the last aof rewrite (100%) auto-aof-rewrite-min-size 64mb # indicates the minimum size of a file when aof is rewrittenCopy the code

Advantages of AOF: Data security, AOF persistence can be configured with the appendfsync attribute and always. Every command operation is recorded in the AOF file once.

Disadvantages of AOF: When the data set is large, it is less efficient to start than RDB

Mixed persistence

Aof-use-rdb-preamble yes

When loading, it first identifies whether the AOF file starts with a REDIS string. If so, it is loaded in the RDB format. After loading the RDB, it continues to load the rest of the file in the AOF format. Hybrid persistence solutions combine RDB speed with AOF security

Pay attention to my technical public number, there are high quality technical articles pushed every week.

Scan the qr code below on wechat to follow: