1, the RDB (Redis Database)

A. Basic concepts

Concept: Write a Snapshot of an in-memory data set to disk at a specified time interval.

Redis forks a separate subprocess for persistence, writing data to a temporary file that will be used to replace the last persistent file after the persistence process is complete. During the whole process, the main process does not perform any IO operations, which ensures high performance. If large-scale data recovery is required and data integrity is not very sensitive, the RDB method is more efficient than the AOF method. The downside of RDB is that data can be lost after the last persistence.



Fork copies the same process as the current one. All data values of the new process (variables, environment variables, program counters, etc.) are the same as those of the original process, but are a new process and a child of the original process.

RDB stores the dump. RDB file

B. How do I trigger the RDB snapshot

1 Cp dump.rdb dump_new.rdb is used to configure the default snapshot in the configuration file after cold copy. RDB will be restored to dump_new. RDB and then loaded automatically on reboot.



② Command save or bgSave, this will force the backup.

  • Save: saveJust to save, block everything else.
  • BGSAVE: Redis asynchronously takes snapshots in the background and responds to client requests. You can run the lastsave command to obtain the time when the last snapshot was successfully executed.
  • Flushall is also generated when the flushall command is executeddump.rdbFile, but it was empty and meaningless (A pit was dug).

Namely: call
saveBackup immediately, immediately.
flushAllYou can make a backup immediately, but it doesn’t make sense.

Redis. conf file about RDB configuration:

  • saveConfiguration: RDB is the compressed Snapshot of the entire memory, RDB data structure,You can configure composite snapshot trigger conditions. The default is 10,000 changes in a minute,

    Or 10 changes in five minutes,

    Or once in 15 minutes.
  • stop-writes-on-bgsave-error: If the value is set to NO, it indicates that you do not care about data inconsistency or other means of discovery and control.
  • rdbcompression: rdbCompression: You can set whether to compress snapshots stored in disks. If so, Redis uses the LZF algorithm for compression. If you don’t want to use CPU for compression, you can turn this feature off.
  • dbfilenameDefault is:dump.rdb.
  • dirGenerated by:dump.rdbThe default directory of.

C. How to recover

Move the backup file (dump.rdb) to the redis installation directory and start the service.

CONFIG GET dir Obtains the directory

D. Advantages and disadvantages

Advantage:

  • Suitable for large-scale data recovery;
  • Low requirements for data integrity and consistency;

Disadvantage:

  • Backups are made at regular intervals, so if Redis unexpectedly goes down, all changes since the last snapshot are lost;
  • When forking, the data in memory is cloned, and roughly twice the expansibility needs to be considered.

E. How to stop

Redis -cli config set save “”

Dump. RDB to restore configuration files

2, AOF(Append Only File)

A. Basic concepts

Each write operation is recorded in a log.

All write commands executed by Redis are recorded (read operations are not recorded), only files can be appended but files can not be overwritten. When Redis is started, the file will be read to rebuild data. In other words, when Redis restarts, the write commands will be executed from front to back according to the contents of the log file to complete data recovery.

Aof holds the appendone.aof file.



B. Configuration position

  • appendonlyThe default isnoWe’re going to changeyesWill be useful;
  • appendfilenameThe default isappendonly.aof;
  • appendfsync:
    • Always: Synchronous persistence Every data change is recorded immediately. Poor disk performance but good data integrity.
    • Everysec: The factory default is recommended. Asynchronous operation is performed every second. If the system is down within one second, data is lost.
    • No;


  • no-appendfsync-on-rewriteAppendfsync (default no) is used to ensure data security.
  • auto-aof-rewrite-min-size: Sets the reference value for rewriting;
  • auto-aof-rewrite-percentage: Sets the reference value for rewriting;



C, AOF start, restore, repair

Normal recovery:

  • Boot: Set Yes, – > change the defaultappendonly noChange to yes.
  • Make a copy of the AOF file with data and save it to the corresponding directory (config get dir);
  • Recovery: Restart Redis and reload.

Abnormal recovery (i.e., sabotage below)

  • Backup AOF file that was written bad;
  • redis-check-aof --fixMake repairs;

Appendone.aof (redis-server) appendone.aof (redis-server)



Step 2: Do this and view the updated appendone.aof file:



This file cannot be used to flushhall, but we can manually edit appendone.aof to restore our database.



However, if our appendone.aof file is corrupted, for example if I randomly add some garbled code, it will cause Redis to fail to start.

Appendonel. aof and dump.rdb can coexist, but appendonel. aof is loaded first.



Redis-check-aof –fix appendone.aof: redis-check-aof –fix appendone.aof



D. rewrite

Concept: AOF uses file apend method, files will become larger and larger to avoid this situation, the new rewrite mechanism, when the size of AOF file exceeds the set threshold, Redis will start the CONTENT compression of AOF file, only retain the minimum instruction set that can recover data. You can use the command bgrewriteaof.

Principle:

If the AOF file continues to grow too large, a new process is forked to rewrite the file (also write temporary files first before rename), traversing the memory of the new process, with a Set statement for each record. The operation of overwriting an AOF file, rather than reading the old AOF file, commands the entire contents of the database in memory to rewrite a new AOF file, similar to a snapshot.

Trigger mechanism:

Redis records the AOF size of the last rewrite, triggered by default when the AOF file size is double the size since rewrite and the file size is greater than 64M.

E. Strengths and weaknesses

Advantage:

  • Sync each change:appendfsync alwaysSynchronous persistence records every data change immediately to disk with poor performance but better data integrity.
  • Synchronization per second:appendfsync everysecAsynchronous operation, recorded every second If the system is down within one second, data is lost.
  • Out of sync:appendfsync noNever synchronized.

Disadvantage:

  • For the data of the same data set, AOF file is much larger than RDB file, and the recovery speed is slower than RDB file.
  • The running efficiency of AOF is slower than that of RDB, and the synchronization efficiency is better than that of RDB.

3. Comparison and selection of RDB and AOF

  • RDB persistence allows you to take snapshots of your data at specified intervals.
  • The AOF persistence mode records each write operation to the server. When the server is restarted, these commands will be executed again to restore the original data. The AOF command saves each write operation to the end of the file using the Redis protocol. Redis can also rewrite AOF files in the background so that AOF files are not too large.
  • Cache only: If you only want your data to live while the server is running, you can also do without any persistence.
  • Open simultaneously:
    • In this case, when Redis restarts, AOF files will be loaded first to recover the original data, because AOF files usually hold more complete data sets than RDB files.
    • RDB data is not real-time, and server restart will only look for AOF files when both are used. Should we just use AOF? The authors advise against it, as RDB is better for backing up databases (AOF is not easy to back up constantly), restarts quickly, and there are no potential AOF bugs left as a back-up measure.


Performance recommendations:


Since the RDB file is only used for backup purposes, it is recommended that the RDB file be persisted only on the Slave and only backed up once every 15 minutes
save 900 1This rule.


If Enalbe AOF,
The benefit is that you lose less than two seconds of data in the worst caseThe startup script is relatively simple. Just load your own AOF file. The cost is constant IO and the inevitable blocking of writing new data to new files in the rewrite process. As long as the hard drive permits,
The frequency of AOF rewrite should be minimizedThe AOF override base size default of 64M is too small and can be set above 5G. The default size exceeds 100% of the original size and overrides can be changed to an appropriate value.


If AOF is not enabled, only master-slave Replication can be used to implement high availability. It saves a lot of IO and reduces the system volatility that comes with rewriting. The trade-off is that if the Master/Slave is dropped at the same time, the data will be lost for more than ten minutes. The startup script will also compare the RDB files in the two Master/Slave files and load the newer one. Sina Weibo has chosen this structure.

Original: www.java520.cn/redis/33.ht…