Redis persistence

preface

Rdis reads and writes are done in memory, so Redis has high performance. Persistence can effectively avoid data loss caused by process exit, and the data can be recovered by using the previous persistent file during the next restart.

Several ways to persist

Redis persists in three ways:

  • Snapshot mode (RDB, Redis DataBase)

RDB persistence is a process in which snapshots of the current process data are generated and saved to disks. RDB persistence can be triggered manually or automatically

  • File Append (AOF, Append Only File)

Record all operation commands and append them to the file as text.

  • Hybrid persistence,

Mixed persistence is a new method added after Redis 4.0. It combines the advantages of RDB and AOF. When writing, the current data is first written to the beginning of the file in the form of RDB, and then the subsequent operation commands are saved to the file in the format of AOF, which can ensure the speed of Redis restart. It also reduces the risk of data loss.

Each of the above three persistence schemes has a specific usage scenario, which we can choose according to our own requirements.

RDB persistence

Redis DataBase (RDB) is a process of writing a Snapshot of a memory ata certain time to disks in binary mode.

RDB can be persisted in two ways:

One is manual trigger, the other is automatic trigger

Manual trigger

Manually triggering Redis provides two commands: save and bgSave.

  • The save command

Manual firing of save blocks the main thread

When the save command is executed by the client, persistence will be triggered, but it will also cause the Redis main thread to block, and the corresponding client command will be executed only after the RDB persistence is completed. It is not recommended to use this command in online environment.

The following figure illustrates using the save command

Start Redis (default configuration is RDB persistence), RDB file time.

Then use Save to persist the data

  • Bgsave command

Bgsave does not block the main thread

When the BGsave command is used, the Redis process forks to create a child process, which is responsible for RDB persistence and ends automatically. Blocking only happens during the fork phase, which is usually very short and rarely blocking, so bgSave is obviously better for persisting data than save.

The following figure shows the start of automatic persistent data

Automatic trigger

How do I trigger RDB persistence automatically?

There is such a configuration in the redis.conf file

What does that mean?

  • save m n

Save m n Indicates that the BGsave command is automatically triggered if the data set has been modified for n times within m seconds.

Save 900 1 indicates that RDB persistence will be triggered if at least one key changes within 900 seconds. Save 300 10 indicates that RDB persistence will be triggered if at least 10 keys change within 300 seconds. Save 60 10000 indicates that RDB persistence will be triggered if at least 10000 keys change within 60 seconds.

If any of these conditions are met, Redis automatically triggers the BGSVAE command for persistence.

  • flushall

Flushhall flushes data in memory. Persistence flushes.rdb files that were previously persisted.

  • The primary and secondary synchronization is triggered

When a slave node performs a full copy operation, the master node executes the BGsave command and sends the RDB file to the slave node. This process automatically triggers Redis persistence.

RDB configuration description

# RDB save conditions
save 900 1
save 300 10
save 60 10000

Whether to stop persisting data to disk after # bgSave fails: yes: stop persisting data to disk, no: ignore the error and continue writing to the file.
stop-writes-on-bgsave-error yes

# RDB file compression
# Redis will use LZF algorithm for compression. If you don't want to consume CPU performance for file compression, you can disable this feature, # the disadvantage is that you need more disk space to store files.
rdbcompression yes

# Enable RDB file check when writing and reading files, check whether there is any damage, if damage is found during startup, stop startup.
rdbchecksum yes

# RDB file name
dbfilename dump.rdb

# RDB file directory
dir ./

Copy the code
RDB pros and cons

advantages

  • The RDB is a compact binary file that represents a snapshot of Redis data at a point in time. It is suitable for backup and full copy scenarios.
  • RDB files can be restarted faster than AOF files.
  • RDB is useful for disaster recovery because it is a compact file that can be transferred to a remote server faster for Redis service recovery

disadvantages

  • RDB data cannot be persisted in real time/second level. RDB can only save data for a certain interval. If Redis fails during this period, data will be lost for a period of time.
  • RDB data cannot be persisted in real time/second level. Because bgSave forks every time it runs, it is a heavyweight operation that is costly to execute frequently.
Disabling persistence
 save ""
Copy the code

AOF persistence

AOF (Append only File) persistence: Each write command is recorded in an independent log, and the commands in the AOF file are executed again when the system restarts to recover data. The main role of AOF is to solve the real-time of data persistence, which has been the mainstream way of Redis persistence

AOF workflow operations

Command write (append), file synchronization (sync), file rewrite (rewrite), load (restart).

Configuration AOF

The default values are “no” and “yes”.

  • Automatic trigger

Persistent configuration

  • Always: Each Redis operation command is written to the disk, and a maximum of one data is lost, which degrades Redis performance. However, data is almost complete and data loss is not a problem.

  • Everysec: Writes data to the disk once every second and loses data for a maximum of one second. The compromise between data access and performance can meet most application scenarios.

  • No: The rule for writing data to disks is not set. The system determines when to write data to disks based on the current operating system.

  • Manual trigger

Aof persistence can be triggered manually using the bgrewriteaof command

As time goes by and the amount of data becomes larger, AOF keeps persisting. As the disk logs become larger and larger, Redis restarts more and more slowly. To solve this problem, Redis provides the AOF rewrite function.

The data in Redis is limited to a certain amount, not exceeding the size of physical memory. It is impossible to say that the data in Redis grows infinitely, leading to the infinite growth of AOF. At a certain point, Redis will use the cache elimination algorithm LRU to automatically clear part of the data from the memory.

AOF overrides configuration

  • Auto-aof -rewrite-min-size: The minimum file size allowed for Aof rewriting. The default is 64 MB.
  • Auto-aof -rewrite-percentage: Percentage of aOF file rewrites. The default value is 100, indicating 100%. Aof rewrites only when the current AOF file is twice the size of the last AOF file.

AOF rewrite process

AOF is where every write command is stored, so it gets bigger and bigger until AOF does rewrite and rewrites a new AOF file.

The advantages and disadvantages

AOF advantages

AOF persistence stores more complete data. AOF provides three retention strategies: Save every operation, save once every second, save with the persistence policy of the system, save once per second, which is a compromise choice from two aspects of data security and performance. It is also the default POLICY of AOF. Even if an accident occurs, the data will be lost for at most 1s.

AOF uses the command append writing mode, so there is no problem of file damage. Even if half of the persistent data of the last operation is written due to some unexpected reasons, it can be easily repaired by the redis-check-aof tool.

AOF persistence files, very easy to understand and parse, are files of all Redis key manipulation commands to disk. Even if you accidentally use the flushall command to delete all key values, you can use the AOF file to delete the last flushall command and restart Redis to restore the deleted data.

AOF shortcomings

AOF files are larger than RDB files for the same data set;

In the case of high Redis load, RDB performs better than AOF.

While RDB uses snapshots to persist the entire Redis data, AOF simply appends each command executed to an AOF file, so IN theory RDB is more robust than AOF.

Mixed persistence

Both RDB and AOF persistence have their own advantages and disadvantages. RDB will cause data loss in a period of time, and AOF files will become larger and larger, which will affect the startup speed of Redis. In order to take into account the advantages of RDB and AOF at the same time, Redis provides mixed persistence mode after version 4.0.

Mixed persistence

When AOF is overwritten, Redis persistent data is written to the beginning of AOF file in RDB format, and the subsequent data is appended to the end of the file in AOF format, as shown in the following figure.

Open the hybrid

Change no to yes

Advantages and disadvantages of mixed persistence

advantages

Hybrid persistence combines the advantages of RDB and AOF persistence. The RDB format allows Redis to start faster, while reducing the risk of massive data loss by combining the advantages of AOF.

Disadvantages:

If the content in RDB format is added to AOF file, the readability of AOF file will be poor and not easy to read. If mixed persistence is enabled, you must use Redis 4.0 and later.

When using mixed persistence, you can choose to turn off RDB or AOF, or turn off persistence depending on your business.

Pay attention to my

Long press qr code

If you like this article, like it, check it out, forward.