Redis has high performance because all reads and writes are done in memory. When our server is disconnected or restarted, the data will disappear, so how do we solve this problem?
In fact, Redis has provided us with a persistence mechanism, respectively RDB and AOF two ways, next let me see how these two tips ensure data persistence.
Persistence Because Redis is a memory-based database, our data is not secure in the event of a server failure.
At this time, we need to store the data in memory to the disk. When the server restarts, we can recover the data through the disk. This process is called Redis persistence.
Redis persistence
Introduction to THE RDB The full name of the RDB is Redis Database Backup file, which can also be called Redis data snapshot.
The RDB file is a compressed binary file (default: dump.rdb);
RDB files are saved on hard disk;
Record database state by saving key-value pairs in the database.
create
When Redis persists, the program saves the current in-memory database state to disk.
create
There are two main Redis commands to create an RDB file: SAVE and BGSAVE.
SAVE synchronization operation. When the command is executed, the Redis server process is blocked and command requests sent by the client are rejected.
Code examples:
def SAVE():
RdbSave ()Copy the codeHere is:
The Save command
BGSAVE is an asynchronous operation in which the child process performs the save work when a command is executed, and the server can continue to let the main thread process command requests sent by the client.
Code examples:
Signal_parent () elif pid > 0: def BGSAVE(): # parent() elif pid > 0: Handle_request_and_wait_signal () else: HANDle_fork_error ()Copy the codeHere is:
BgSave command
load
The load is performed automatically when the server starts.
load
The server blocks while loading the RDB file until the load is complete.
Main Setup Redis allows users to have the server automatically execute BGSAVE commands at regular intervals by setting the save option for the server configuration.
Set save conditions The following configurations are provided:
save 900 1
save 300 10
Copy the codeIn this case, the BGSAVE command is executed as long as one of the following conditions is met:
The server made at least one change to the database within 900 seconds;
The server made at least 10 changes to the database within 300 seconds.
The SaveParams server program sets the saveParams property of the server state redisServer structure based on the save conditions set by the Save option.
The saveParams property is an array; Each element in the array is a saveParam structure; Each saveParam structure holds a save condition set by the Save option. Struct saveparam {time_t seconds; Int changes; }Copy the codeDirty The dirty counter records how many times the server has changed the database state (including writes, deletes, and updates) since the last SAVE or BGSAVE command was successfully executed.
Lastsave is a UNINX timestamp that records the last time the server successfully executed the SAVE or BGSAVE command.
The server periodic operation function serverCron (which maintains the running server) executes this function every 100 milliseconds by default. One of its tasks is to check whether the save criteria set by the save option have been met, and then execute the BGSAVE command.
Code examples:
def serverCron(): # .... Saveparam in server. Saveparams: Save_interval = unixtime_now() -server.lastsave # if the state of the database has been changed more times than the condition set # if the time since the lastsave exceeds the condition set if server.dirty >= saveparam.changes and save_interval > saveparam.seconds: BGSAVE()Copy the codeDefault Configuration Default configuration of the RDB file is as follows:
################################ SNAPSHOTTING ################################ # # Save the DB on disk: Automatic persistence for a given number of seconds and a given number of write operands to the database. # save <seconds> <changes> # save 900 1 save 300 10 save 60 10000 Yes stop-writes-on-bgsave-error yes # Use LZF to compress string objects for persistence. Rdbcompression yes # Whether to checksum RDB files, usually yes rdbchecksum yes # RDB persistent filename dbfilename dump. RDB # Persistent file storage directory dir./Copy the codeAOF
Introduction to the
AOF stands for Append Only File. A log is a write-after log. A Redis command is executed to write data to the memory before logging.
By saving the write command executed by Redis server to record the database state;
All commands written to AOF files are saved in Redis command request protocol format.
implementation
AOF persistence process is mainly implemented through the following processes:
AOF process
Command appending If AOF persistence is enabled, the server appends the write command to the end of the aOF_buf buffer of the server state in protocol format after executing a command.
The file synchronization server calls flushAppendOnlyFile every time it terminates an event loop, which considers whether the contents of the aOF_buf buffer need to be written to and saved to an AOF file.
The flushAppendOnlyFile function performs the following process:
- WRITE: Writes the cache in aOF_buf to the AOF file according to the condition;
- SAVE: Depending on the condition, call the fsync or fdatasync function to SAVE the AOF file to disk.
This function is influenced by the three appendfsync values configured by the server: always, Everysec, and no, also known as the three policies.
Always
Each command is fsync to the hard disk so redis doesn’t lose its write data.
Always
everysec
The buffer is flushed to hard disk every second (default).
everysec
no
It is not up to us to decide when to flush to hard disk based on the rules of the current operating system.
no
The data load
Create a pseudo client without network connection.
Analyze and read a write command from AOF file;
Use pseudo client to execute read write command;
Continue steps 2 and 3 until all write commands in the AOF file have been processed.
File rewriting Why file rewriting is required:
- To solve the problem of AOF file volume expansion;
- Replace the existing AOF file by rewriting it to create a new AOF file that does not contain any redundant commands that waste space.
Implementation file rewrite implementation principle:
- There is no need to do anything with existing AOF files;
- Read the current value of the key directly from the database;
- Record the key-value pair with a single command instead of multiple commands that record the key-value pair.
The background override does not block the parent process, and Redis executes the AOF override in the child process.
During AOF rewrite by the child process, the server process needs to perform three processes:
- Execute the command sent by the client.
- Appends the executed write command to the AOF buffer;
- Appends the executed write command to the AOF rewrite buffer.
Server flow
Default Configuration The default configuration of the AOF file is as follows:
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # APPEND ONLY MODE # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # open appendonly AOF persistent way no AOF persistence file name Appendfilename "appendonly. Aof "# fsync buffer data to disk every second appendfsync everysec # appendfsync no # Whether data is not synchronized to aOF file during rewrite No-appendfsync-on-rewrite no # Indicates the rate at which AOF file rewrites are triggered. Auto-off-rewrite-percentage 100 # Indicates the minimum size at which AOF file rewrites are triggered Aof-load-truncated yes # Whether to turn on the mixing switch aof-use-rdb-preamble yesCopy the codeSummary through the above introduction, we must have a general understanding of Redis persistence, so these two ways, how should we choose?
For large and medium-sized applications, we should use RDB and AOF in combination to ensure data integrity and high efficiency.
AOF is preferred to ensure data integrity and prevent data loss.
If you are dealing with large-scale data recovery and pursuing higher and faster efficiency, RDB is preferred.
You can also refer to the following figure for selection:
