Introduction to AOF persistence

AOF persistence (Append Only File) records each write operation in the form of a log, and records all write instructions performed by Redis (read operations are not recorded, similar to SQL files that record write operations when backing up database data). Only AOF files can be appended but files cannot be overwritten.

When Redis starts, it reads this file to rebuild the data. In other words, redis restarts and executes write instructions from front to back based on the contents of the log file to complete data recovery.

AOF persistent configuration

By default, the AOF persistence data is stored in an AOF file named appendone. AOF in the directory configured for dir, and the persistent-related configuration file information is stored in the APPEND ONLY MODE module of redis. Redis Configuration file for more information

AOF persistence is disabled by default
appendonly no
Appendone.aof = appendone.aof
appendfilename "appendonly.aof"
Always mode, always write to aOF files, complete disk synchronization, poor performance but good data integrity
# appendfsync always
Everysec writes to aOF files every second and completes disk synchronization. If it goes down within 1 second, the last second of data will be lost, as opposed to always and No
appendfsync everysec
# Persistence policy no mode, write aOF file, do not wait for disk synchronization, the fastest
# appendfsync no
[root@appendfsync] [root@appendfsync] [root@appendfsync] [root@appendfsync
no-appendfsync-on-rewrite no
Redis can call bgreWriteAOF to rewrite the log file when the aOF file size exceeds the specified percentage of the size of the last rewritten AOF file.
auto-aof-rewrite-percentage 100
# The minimum aOF file size allowed to be overwritten
auto-aof-rewrite-min-size 64mb
Copy the code

Note: The bgrewriteAOF mechanism overwrites aOF in a child process, so that the main process does not block the rest of the command processing, and also solves the problem of aOF file size. The bgreWriteAOF operation and the main process write aOF file at the same time, both of them operate on disk, and bgreWriteAof usually involves a lot of disk operations, which causes the main process to block writing aOF file. Now the no-appendfsync-on-rewrite argument comes in. If this parameter is set to no, it is the safest way not to lose data, but to suffer from blocking problems. What if I set it to yes? This is equivalent to setting appendfsync to no, which means that no disk operation is performed, just writing to the buffer, so there is no blocking (because there is no competing disk), but if Redis fails at this point, the data will be lost. How much data is missing? Under the default Settings of the Linux operating system, a maximum of 30 seconds of data is lost. Therefore, if the application system cannot tolerate delays and can tolerate a small amount of data loss, set it to yes. If can’t stand the loss of data, application system is set to no reference here: www.cnblogs.com/ExMan/p/102…

Restore data

  1. Enable AOF persistence.

Appendonly yes 2. Set the AOF file name. The default name is appendone.aof. appendfilename “appendonly.aof” 3. Configure the AOF file in which to recover data. The default directory is the path where the redis service is enabled. Dir./ 4. Put the AOF file with the name of appendfilename in the dir directory. When the Redis service starts, the command of AOF file record is automatically executed to restore the data.

Fix AOF file

If the AOF file fails to be restored due to an exception, run the redis-check-aof –fix <AOF file name > command to restore the file.

advantages

  1. Even in the worst case, the different appendfsync policies ensure that no more than 2 seconds of data is lost.
  2. When an AOF file exceeds a certain threshold, a rewriting mechanism is triggered to ensure that the file is not too large.

disadvantages

  1. For the data of the same data set, AOF files are much larger than RDB files, and the recovery speed is slower than RDB.
  2. The running efficiency of AOF is slower than that of RDB, and the synchronization efficiency is better than that of RDB.
  3. Aof file rewriting and Redis data recovery can cause blocking problems even if there is a rewriting mechanism in AOF.

conclusion

  1. The AOF file is an append only log file. Redis can automatically rewrite the AOF in the background if the AOF file becomes too large.
  2. AOF files keep all the writes to the database in an orderly fashion in the redis protocol format, making the contents of AOF files easy to read and analyze.
  3. AOF files are usually larger than RDB files for the same data set, and depending on the fsync strategy used, AOF may be slower than RDB.

RDB and AOF selection

Since RDB files are only used for backup purposes, it is recommended to persist RDB files only on the Slave and backup them only once every 15 minutes, leaving only the save 9001 rule.

If you Enalbe AOF, the benefit is that you lose less than two seconds of data in the worst case. The startup script is simpler and you just load your own AOF files. The cost is constant IO and the inevitable blocking of writing new data to new files in the rewrite process. The frequency of AOF rewrite should be minimized as long as hard drives are permitted; the default base size of AOF rewrite, 64M, is too small and can be set to more than 5G. The default size exceeds 100% of the original size and overrides can be changed to an appropriate value.

If AOF is not enabled, only master-slave Replication can be used to implement high availability. It saves a lot of IO and reduces the system volatility that comes with rewriting. The trade-off is that if the Master/Slave is dropped at the same time, the data will be lost for more than ten minutes. The startup script will also compare the RDB files in the two Master/Slave files and load the newer one. Sina Weibo has chosen this structure.