This article is mainly my reading experience after reading “Redis Design and Implementation”, I hope it can help you, thank you!
Chapter 15 Reproduction
In Redis, you can have the slave server back up data on the master server by executing the SLAVEOF command or setting the SLAVEOF option. Redis replication functions are mainly divided into synchronization and command propagation. Synchronization mainly refers to updating the state of the slave server to that of the master server. Synchronization is further broken down into full and partial resynchronization (available only in Redis 2.8 and later). ##### Full resynchronization usually sends command requests from the server to the master server. The master server sends write commands saved in the RDB file and buffer to the slave server. The slave server restores the database status based on the write commands saved in the RDB file and buffer. The procedure is as follows: 1) The secondary server sends the SYNC command to the primary server. 2) The primary server receiving SYNC executes the BGSAVE command, generates an RDB file in the background, and uses a buffer to record all write commands executed from now on. 3) When the BGSAVE command is executed by the master server, the master server sends the RDB file generated by the BGSAVE command to the slave server. The slave server receives and loads the RDB file to update its database state to the database state of the master server when the BGSAVE command is executed. 4) The master server sends all write commands recorded in the buffer to the slave server, and the slave server executes these write commands to update its database state to the current state of the master server database. As shown below:
Implementation of the PSYNC command
PSYNC has two modes for full and partial resynchronization. Will the slave server send PSYNC to the master server when starting a new replication? -1 command to proactively request full resynchronization from the primary server. If the slave server has already replicated a master server, the slave server sends the PSYNC command to the master server when starting a new replicate: Where runid is the runid of the primary server from which the last replication was performed, and offset is the current replication offset from the secondary server. The primary server receiving this command uses these two parameters to determine which synchronization operation should be performed on the secondary server. The specific flow chart is as follows:
Implementation of replication
Sending the SLAVEOF command to the slave service enables a slave server to replicate the master service. For example, send the following command to the secondary server to copy the data of the primary server whose address is 127.0.0.1 and port is 6379.
SLAVEOF 127.0.0.1 6379
Copy the codeThe steps are as follows: 1. On the secondary redisServer, set the master server address masterhost and port masterport
Char *masterhost; // Master server port int masterport; };Copy the code2. Establish a Socket connection 3. Send the PING command 4. Send port information 6. Synchronize 7. Command transmission, heartbeat detection, detection of the network connection status of the master and slave servers, “auxiliary realization of Min-slaves configuration options, detection command loss.
Chapter 16 the Sentinel
Sentinel system refers to the sentinel system composed of one or more sentries instance can monitor any number of the primary and subordinate all from the server, the monitoring of the main server into the offline state, an upgrade from the server will automatically give priority to the server, and service instead of the old by the new Lord master server requests continue to process orders. ##### Start sentinel server The sentinel server essentially searches a Redis server running in a special mode. The implementation of sentinel server start is mainly divided into three steps: 1. Initialize the server. Unlike normal Redis servers, there is no need to load an RDB file or AOF file to restore the database state during initialization. The main functions of the Sentinel server are shown below:
Struct sentinelState {// Uint64_t current_epoch; Dict *masters is a pointer to the sentinelRedisInstance structure; dict *masters is a pointer to the sentinelRedisInstance structure. // Is TILT mode enabled? int tilt; // The number of scripts currently being executed int running_scripts; Mstime_t tilt_start_time; // The last execution time of the processor mstime_t previous_time; // a FIFO queue containing all user scripts that need to be executed. } sentinel;Copy the codeA sentinelRedisInstance structure that holds information about the master server being monitored
Typedeft struct sentinelRedisInstance {// Int flags; typedeft struct sentinelRedisInstance; // The name of the primary server is set by the user in the configuration file. // The name of the secondary server and Sentinel is automatically set by Sentinel. // The format is IP :port, for example"127.0.0.1:26379"char *name; Char *runid; Uint64_t config_epoch; // Address of the instance sentinelAddr *addr; // the value of the SENTINEL down-after-milliseconds option // How many milliseconds before an instance does not respond is considered subjective offline (subjectively down). // The number of votes needed to judge the instance as objectively down int quorum; Int parallel_syncs; // The value of the SENTINEL parallel-syncs <master-name> <number> option // SENTINEL failover-timeout <master-name> < MS > value // The maximum time for refreshing the failover status mstime_t failover_timeout; / /... } sentinelRedisInstance;Copy the code4. Create a network connection to the primary server. The Sentinel server creates two asynchronous network connections to the monitored master server and one command connection to send commands to the master server and receive command replies. The other is the subscription connection, which is used to subscribe to the main server’s __sentinel__: Hello channel. Through this channel, the Sentinel server can learn about other sentinel servers, slave servers, and so on from the master server.
Command connection – Get master server information
The Sentinel server sends the INFO command to the monitored master server via a command link every ten seconds and retrits the master server’s current information based on the response. You can obtain the following information: 1. Information about the primary server (such as the running ID and role) 2. Based on this information, sentinel objects can update master and slave information in their own sentinelRedisInstance structure. (Can automatically discover slave server information)
Command connection – Gets information about the slave server
When the sentinel object detects that a new slave appears, it creates an instance structure for it, as well as command and subscription connections to the slave. The INFO command is sent to the slave server over the command connection every ten seconds to get information about the slave server and its master server. The following information is obtained: Running ID and role of the secondary server. IP address of the primary server, port number master_port, connection status of the primary server and secondary server, priority of the secondary server slave_priority, replication offset of the secondary server slave_repl_offset The following figure shows what happens after updating the slave server instance structure based on the INFO command reply above:
Send information to master and slave servers
By default, Sentinel sends PUBLISH to all monitored primary and secondary servers by command connection every two seconds in the following format:
PUBLISH __sentinel__:hello "<s_ip>,<s_port>,<s_runid>,<s_epoch>,<m_name>,<m_ip>,<m_port>,<m_epoch>
Copy the codeParameters include sentry server IP, port, run ID, configuration era. Name, IP, port, configuration era of the primary server. Here is an example of the message Sentinel sends to the primary server via the PUBLISH command:
"127.0.0.1, 26379, e955b4c85598ef5b5f055bc7ebfd5e828dbed4fa, 0, mymaster, 127.0.0.1, 6379, 0Copy the codeThis sample contains the following information: the IP address of the Sentinel, who is 127.0.0.1 port number is 26379, running with ID e955b4c85598ef5b5f055bc7ebfd5e828dbed4fa, configuration of the current era of 0. · The name of the primary server is MyMaster, the IP address is 127.0.0.1, the port number is 6379, and the current configuration era is 0.
Channel information received from master and slave servers
When Sentinel establishes a subscription connection with a master or slave server, Sentinel sends the following command to the server via the subscription connection:
SUBSCRIBE __sentinel__:hello
Copy the codeSentinel’s subscription to the __sentinel__: Hello channel continues until Sentinel is disconnected from the server. For each server connected to Sentinel, Sentinel both sends messages to the server’s __sentinel__: Hello channel through a command connection and receives messages from the server’s __sentinel__: Hello channel through a subscription connection. When Sentinel1 sends a message to the server’s __sentinel__: Hello channel, all sentinels subscribed to the __sentinel__: Hello channel (including Sentinel1 itself) receive the message, The corresponding master server instance structure is then updated.
Update the Sentinels Dictionary
The Sentinels dictionary in the Sentinel instance structure created for the master server holds information about all sentinels that also monitor the master server, in addition to Sentinel itself. When a Sentinel receives a message from another Sentinel, the message is parsed and updated to the Sentinels dictionary.
· The sentinels dictionary key is the name of one of the sentinels in the format IP :port. For example, Sentinel at IP address 127.0.0.1 and port 26379, The key to Sentinel in the Sentinels dictionary is “127.0.0.1:26379”. · The sentinels dictionary value is the Sentinel instance structure corresponding to the key. For example, the sentinels dictionary value for the key “127.0.0.1:26379” is the Sentinel instance structure with IP 127.0.0.1 and port 26379. ##### “Create command links to other Sentinels When Sentinel finds a new Sentinel through channel information, it not only creates the corresponding instance structure in the Sentinels dictionary for the new Sentinel, but also creates a command link to the new Sentinel. The new Sentinel will also create command connections to this Sentinel, and eventually multiple Sentinels monitoring the same master server will form an interconnected network: Sentinel A has A command connection to Sentinel B, and Sentinel B also has A command connection to Sentinel A, as shown in the following figure:
Detect subjective offline status
By default, Sentinel sends a PING command once per second to all instances (primary, secondary, and other Sentinels) with which it has created command connections, and determines whether the instance is online by the PING response returned by the instance. · Effective reply: the instance returns one of the three replies: +PONG, -loading, or -masterDown. · Invalid reply: the instance returns replies other than +PONG, -loading, and -masterDown, or does not return any replies within the specified time limit. The down-after-Milliseconds option in the Sentinel profile specifies the length of time it takes for a Sentinel judgment instance to go subjective offline: If an instance returns an invalid response to Sentinel in back-to-back up-after-milliseconds, Sentinel will change its flags to enable SRI_S_DOWN. This indicates that the instance has gone subjectively offline.
Detect objective offline status
To determine if a primary server is actually offline, Sentinel sends the “Sentinel IS-master-down-by-addr” command to other Sentinels that also monitor the primary server. See if they agree that the master server has gone offline (either subjectively or objectively). When Sentinel receives a sufficient number of offline judgments from other Sentinels, Sentinel determines that the slave server is objectively offline and performs a failover on the primary server. There is a quorum attribute in the Sentinel configuration. When the Sentinel with quorum number considers that the master service is in the offline state, Sentinel will take the master server to determine the customer offline. (Each Sentinel can have a different quorum).
Election Lead Sentinel
When a primary server is judged to be objective offline, the sentinels monitoring the offline primary server negotiate to elect a lead Sentinel, who will perform failover operations on the offline primary server. Here are the rules and methods for Redis election lead Sentinel: 1. Each Sentinel (source Sentinel) sends the Sentinel IS-master-down-by-addr command to the other Sentinel (target Sentinel), asking the latter to set the former as the local lead Sentinel. Each Sentinel setting local lead Sentinel rule is first-come, first-served. The source Sentinel that first sends setting requirements to the target Sentinel will become the local lead Sentinel of the target Sentinel, while all subsequent setting requirements received will be rejected by the target Sentinel. 2. Once the local lead is set, the target Sentinel will configure epoch +1, and reply the operation ID and configuration epoch of the local lead Sentinel to the source Sentinel. 3. After receiving the command reply from the target Sentinel, the source Sentinel will check whether the value of the leader_EPOCH parameter in the reply is the same as its own configuration era. If so, the source Sentinel will continue to fetch the leader_runid parameter in the reply. If the value of the Leader_runid parameter is the same as the run ID of the source Sentinel, then the target Sentinel has set the source Sentinel to the local lead Sentinel. 4. If a Sentinel is set as the local lead Sentinel by more than half of the sentinels, the Sentinel becomes the lead Sentinel. 5. If no Sentinel is elected as the Lead Sentinel within a given time limit, each Sentinel will be elected again at a later time until the lead Sentinel is elected.
failover
After the lead Sentinel is elected, the lead Sentinel will perform a failover operation on the offline primary server, which consists of the following three steps: 1. Out of all slave servers under the offline master, select one slave server and convert it to the master server. Selection rules: Delete all secondary servers that have not returned the Sentinel INFO command in the last five seconds. Delete all secondary servers that have been disconnected from the primary server for more than down-after-milliseconds10 milliseconds. The down-after-milliseconds option specifies how long it takes to determine how long a primary server takes offline. Removing secondary servers that are longer than down-after-milliseconds10 milliseconds ensures that all secondary servers in the list haven’t disconnected too early. The rest of the data stored from the server in the list is relatively new. The lead Sentinel then sorts the remaining slave servers in the list according to their priority and selects the highest-priority slave server. 2. The lead Sentinel sends the SLAVEOF command to the slave server to copy all slave servers under the offline master server to the new master server. 3. When the old master server comes back online, the lead Sentinel sends it the SLAVEOF command to set the offline master server as the slave server of the new master server.
Chapter 17 cluster
Redis clustering is a distributed database solution that can be sharded for data sharing and provides replication and failover capabilities.
node
A node is just a Redis server running in cluster mode, determined by the cluster-Enabled property in the startup configuration.
Cluster data structure
The clusterNode structure can be used to store the current status of a node, such as creation time, name, configuration era, IP, port, etc.
Record the information about all slots assigned to a cluster
In addition to recording the slots of its own nodes, the Slots array in the clusterState structure records the assignment information of all 16384 slots in the cluster.
def slot_number(key):
return CRC16(key) & 16383
Copy the codeThe CRC16 (key) statement is used to calculate the CRC-16 checksum of the key, and the &16383 statement is used to calculate an integer between 0 and 16383 as the slot number of the key. ##### “node database implementation Cluster nodes store key-value pairs and key-value pair expiration time in exactly the same way as stand-alone Redis server stores key-value pairs and key-value pair expiration time. One difference between a node and a stand-alone server in terms of database is that the node can only use database 0. The following figure shows the database status of node 7000. The database contains the list key “LST”, hash key “book”, and string key “date”, where key “LST” and key “book” have expired time.
The shard
The Redis cluster resharding operation can change any number of slots assigned to one node (the source node) to another node (the target node), and the key-value pairs of the associated slots are moved from the source node to the target node. Resharding can take place online. During resharding, the cluster does not need to go offline, and both source and target nodes can continue to process command requests. Resharding is performed by Redis’s cluster management software, Redis-Trib. Redis provides all the commands required for resharding, while Redis-Trib sends commands to source and target nodes. The steps for redis-trib to refragment a single slot in the cluster are as follows: 1) Redis-trib claims that the target node is porting <source_id> with the CLUSTER SETSLOTIMPORTING<source_id> command; the target node is porting from the source node to import key pairs belonging to slots. When migrating <target_id>, redis-trib can perform CLUSTER SETSLOTMIGRATING<target_id> on a source node to prepare the source node to migrate key-value pairs from slot to target node. 3) Redis-trib sends the CLUSTER GETKEYSINSLOT command to the source node to obtain a maximum of count key names of the slot key-value pairs. Resharding is performed by Redis’s cluster management software, Redis-Trib. Redis provides all the commands required for resharding, while Redis-Trib sends commands to source and target nodes. The steps for redis-trib to refragment a single slot in the cluster are as follows: 1) Redis-trib claims that the target node is porting <source_id> with the CLUSTER SETSLOTIMPORTING<source_id> command; the target node is porting from the source node to import key pairs belonging to slots. When migrating <target_id>, redis-trib can perform CLUSTER SETSLOTMIGRATING<target_id> on a source node to prepare the source node to migrate key-value pairs from slot to target node. 3) Redis-trib sends the CLUSTER GETKEYSINSLOT command to the source node to obtain a maximum of count key names of the slot key-value pairs. 4) For each key name obtained in Step 3, redis-trib sends a MIGRATE<target_ip><target_port><key_name>0 to the source node to MIGRATE the selected key atomically from the source node to the target node. 5) Repeat Step 3 and Step 4 until the source node migrates all key-value pairs that belong to slots to the destination node. Figure 17-24 shows the key migration process. 6) Redis-trib sends the CLUSTER SETSLOTNODE<target_id> command to any node in the CLUSTER to assign slot slots to the target node. This assignment information will be sent to the entire CLUSTER via message. Eventually all nodes in the cluster will know that slots have been assigned to target nodes.
typedef struct clusterState {
clusterNode *importing_slots_from[16384];
clusterNode *migrating_slots_to[16384];
} clusterState;
Copy the codeIf the value importing_SLOts_from [I] is not NULL and points to a clusterNode structure, It indicates that the current node is importing slot I from the node represented by clusterNode. Similarly, the migrating_SLOts_TO array records the slot where the current node is being exported to another node. When the cluster is resharded, the migrating_SLOts_to array is updated by the source node and the importing_SLOts_FROM array is updated by the destination node.
#####ASK error During the process of resharding, when the source node migrates a slot to the target node, it may occur that some key-value pairs belonging to the migrated slot are stored in the source node, while the other key-value pairs are stored in the target node. When a client sends a command related to a database key to the source node, and the database key to be processed by the command happens to belong to the slot being migrated: · The source node first looks for the specified key in its own database, and if it finds it, executes the command sent by the client. , on the contrary, if the source node failed to find the specified key in its own database, then this key may have been moved to the target nodes, source nodes will be returned to the client a ASK error, to guide the client to the target node is being imported, before send the target node wants to follow the orders before, need to send have a first order, Enable the REDIS_ASKING icon on the client. Otherwise, the target node will not run commands for the slot that is being migrated. (The REDIS_ASKING button on the client is a one-time button. The REDIS_ASKING button on the client is removed after the node executes a command sent by the client with the REDIS_ASKING button.) ASK
Fault detection
Each node in the cluster periodically sends PING messages to other nodes in the cluster to check whether they are online. If the node that receives the PING message does not return the PONG message to the node that sent the PING message within the specified time, The sending node will then flag the receiving node as suspected to be offline (PROBABLE fail, PFAIL). Each node in a cluster sends messages to each other to exchange information about the status of each node in the cluster, such as whether a node is online, suspected to be offline (PFAIL), or offline. When primary node A learns that primary node B considers primary node C to be in the suspected offline state through A message, primary node A will find the clusterNode structure corresponding to primary node C in its ClusterState. nodes dictionary. Add the failure report of primary node B to the fail_Reports linked list of the clusterNode structure
Chapter 18 Publishing and Subscribing
In Redis, a client can subscribe to one or more channels, and when other clients send messages to that channel, all subscribers can receive the message. SUBSCRIBE to a channel. PSUBSCRIBE is to subscribe to one or more channels that conform to the rules. PUBLISH is to PUBLISH a message to a channel. When a client executes a SUBSCRIBE command to SUBSCRIBE to a channel or channels, Redis stores all the channels’ subscriptions in the redisServer pubsub_Channels dictionary. The key of the dictionary is the channel that is subscribed to. The value of the key is a linked list of all the clients that subscribe to the channel, as shown in the figure below
Subscribe and unsubscribe patterns
The server stores all channel subscriptions in the server state’s Pubsub_Channels property. Similarly, the server stores all pattern subscriptions in the server state’s Pubsub_Patterns property, which is a linked list. Each node in the linked list contains a Pubsub Pattern structure whose Pattern property records the subscribed Pattern and whose client property records the subscribed Pattern clients. As shown below:
“. · Client client-8 is in subscription mode “book “.
Viewing Subscription Information
The PUBSUB command is one of the new Redis 2.8 commands that allows clients to view information about a channel or mode, such as how many subscribers a channel currently has, or how many subscribers a mode currently has. PUBSUB CHANNELS[pattern] The PUBSUB CHANNELS[pattern] command is used to return the CHANNELS currently subscribed to by the server. PUBSUB NUMSUB PUBSUB NUMSUB[channel-1 channel-2…channel-n] subcommand takes any number of channels as input arguments and returns the number of subscribers to those channels. PUBSUB NUMPAT The PUBSUB NUMPAT subcommand is used to return the number of subscribed modes on the server. This subcommand is implemented by returning the length of the pubsub_Patterns list, which is the number of clients in the server that subscribe to channels using the subscription pattern.
Chapter 19 Affairs
Redis can use transactions to package multiple commands into a single command and execute them once and sequentially on the server without interrupting the transaction to execute command requests from other clients. A transaction starts with a MULTI command, puts multiple commands into the transaction, and commits the transaction to the server by EXEC:
redis> MULTI
OK
redis> SET "name" "Practical Common Lisp"
QUEUED
redis> GET "name"
QUEUED
redis> SET "author" "Peter Seibel"
QUEUED
redis> GET "author"
QUEUED
redis> EXEC
1) OK
2) "Practical Common Lisp"
3) OK
4) "Peter Seibel
Copy the code#### implementation of a transaction ##### Start of a transaction The execution of the MULTI command marks the start of a transaction. The MULTI command can switch the client executing the command from the non-transactional state to the transactional state. This is done by changing the REDIS_MULTI flag in the Flags property of the redisClient. ##### Command queue When a client is in the non-transaction state, the commands sent by the client are immediately taken into account. After a client switches to the transaction state, the server performs different operations based on the commands sent by the client: · If the client sends one of the four commands EXEC, DISCARD, WATCH, MULTI, then the server executes this command immediately. · In contrast, if the client sends a command other than EXEC, DISCARD, WATCH, MULTI, the server does not execute the command immediately. Instead, the server places the command in a transaction queue and returns QUEUED to the client. The process for the server to determine whether the command should be queued or executed immediately is shown in the following figure:
Typedef struct redisClient {// transaction state multiState mstate; /* MULTI/EXEC state */ // ... } redisClient; Typedef struct multiState {// Transaction queue, FIFO order multiCmd *commands; Int count; } multiState; // A transaction queue is an array of type multiCmd. Each multiCmd structure in the array holds information about a queued command, including a pointer to the implementation function of the command, the command's parameters, and the number of parameters: Typedef struct multiCmd {// robj **argv; // Number of arguments int argc; Struct redisCommand * CMD; } multiCmd;Copy the codeThe transaction queue holds queued commands in a first-in-first-out (FIFO) fashion, with the first queued commands placed at the front of the array and the last queued commands placed at the back of the array. ##### Executing transactions When a client in transactional state sends an EXEC command to the server, the EXEC command is immediately executed by the server. The server iterates through the client transaction queue, executes all the commands stored in the queue, and returns all the results of executing the commands to the client. The WATCH command is an optimistic locking. It can monitor any number of database keys before the EXEC command is executed. When the EXEC command is executed, it checks whether at least one of the monitored keys has been modified. The server refuses to execute the transaction and returns an empty reply to the client indicating that the transaction failed. Optimistic and pessimistic locking Pessimistic locking assumes the worst case scenario. Every time data is fetched, it is assumed that the data will be modified, so the lock is locked. In this way, other threads will block until the lock is acquired, and then the data is fetched. Exclusive locks such as Synchronized and ReentrantLock in Java are implementations of the pessimistic locking idea. Optimistic locking is to assume the best case scenario, every time the data is fetched, it is assumed that the data will not be modified, and only when the data is actually modified, it will judge whether the value obtained before is consistent with the current value of the variable in memory. Only when the value is consistent, it will be updated, otherwise it will not be updated. This is usually implemented using version numbers or the CAS algorithm (CAS is the CAS algorithm that atomically updates the value in memory with the new value if and only if the current value in memory is equal to the expected value, otherwise no operation is performed (compare and replace is an atomic operation). In general, it is a spin operation, i.e., continuous retries.) Optimistic locks are suitable for low write scenarios, pessimistic locks are suitable for high write scenarios. ABA problem: If A variable is set to B and then to A, the compareAndSet of JDK1.5 AtomicStampedReference determines whether the object references are the same, and then updates the argument. 2. If the execution fails, the system tries again. If the execution continues for a long time, the CPU will be under pressure. Only atomic operations on one shared variable can be guaranteed. CAS is invalid for multiple shared variables. However, after JDK 1.5, the AtomicReference class is provided to ensure atomicity between reference objects. Multiple variables can be placed in the same object to ensure atomicity. ##### Use the WATCH command to monitor database keys Each Redis database holds a watched_keys dictionary. The key of the watchis dictionary is a database key monitored by the WATCH command, and the value of the dictionary is a linked list of all the clients that monitor the corresponding database key.
Typedeft struct redisDb {// Dict *watched_keys; } redisDb;Copy the codeAn example watched_keys dictionary is shown below. · Clients C1 and C2 are monitoring key “name”. · Client C3 is monitoring the key “age”. · Client C2 and C4 are monitoring the key “address”.
Trigger of monitoring mechanism
All commands that modify the database, such as SET, LPUSH, SADD, ZREM, DEL, FLUSHDB, etc., call multi-. c/touchWatchKey to check watched_keys. The touchWatchKey function turns on the REDIS_DIRTY_CAS flag of the client that is monitoring the changed key, indicating that the transaction security of the client has been broken.
Determine if the transaction is secure
When the server receives an EXEC command from a client, the server decides whether to execute a transaction based on whether the client has the REDIS_DIRTY_CAS flag turned on: If client REDIS_DIRTY_CAS logo has been open, so that the client of the key monitoring, at least one key has been modified, in this case, the client committed transaction is no longer safe, so the server will refuse to enforce the client to submit the transaction, otherwise the server will execute the client committed transaction. ##### ACID properties of transactions In traditional relational databases, ACID properties are often used to test the reliability and security of transaction functions. In Redis, transactions always have Atomicity, Consistency and Isolation, and they also have Durability when Redis runs in a particular persistence mode.
atomic
Atomicity of a transaction means that the database executes multiple operations in a transaction as a whole, and the server either executes all or none of the operations in the transaction. For Redis transactions, all or none of the commands in the transaction queue are executed, so Redis transactions are atomic. The biggest difference between Redis transactions and traditional relational database transactions is that Redis does not support rollback. Even if a command in the transaction queue fails during execution, the entire transaction will continue to execute until all commands in the transaction queue have been executed. (The author believes that Redis transaction execution errors are usually caused by programming errors, so it is not necessary)
consistency
Transaction consistency means that if the database is consistent before the transaction is executed, it should remain consistent after the transaction is executed, regardless of whether the transaction succeeds. Redis transaction consistency is mainly concerned with resolving errors and preventing transaction execution from corrupting the database. Such as enqueue error, execution error, server down. Server downtime If the Redis server is down during the execution of a transaction, the following may occur depending on the persistence mode used by the server: · If the server is running in non-persistent memory mode, the database will be blank after the restart, so the data is always consistent. · If the server is running in RDB mode, downtime in the middle of a transaction will not cause inconsistencies, as the server can restore data from existing RDB files to restore the database to a consistent state. If no RDB file is found available, the database will be blank after a restart, and blank databases are always consistent. · If the server is running in AOF mode, downtime in the middle of a transaction will not cause inconsistencies, as the server can restore the data from existing AOF files, thus restoring the database to a consistent state. If no AOF files are found available, the database will be blank after a restart, and blank databases are always consistent. To sum up, no matter what persistence mode the Redis server is running in, an outage in the middle of a transaction does not matter.
Isolation,
The isolation of transactions means that even if multiple transactions are executed concurrently in the database, they do not affect each other, and that transactions executed concurrently and sequentially produce exactly the same results. Because Redis executes transactions (and commands in transaction queues) in a single-threaded manner, and the server guarantees that the transaction will not be interrupted during the execution of the transaction, Redis transactions are always run in serial fashion, and transactions are always isolated.
The durability of
Transaction durability means that when a transaction is completed, the results of the execution of the transaction have been saved to permanent storage medium (such as hard disk), even if the server is down after the completion of the transaction, the results of the execution will not be lost.
Chapter 21 Sort
Use the SORT command in Redis to SORT the values of list keys, set keys, or ordered set keys.
Implementation of the SORT command
The SORT command creates an array of the same length as the key for each sorted key. Each item in the array is a redisSortObject structure. The obj pointer points to a single element in the list or collection. Sort arrays by score (score is used to store score values, rather than taking the value of an element, because it involves sorting strings and other complications). Here is the complete definition of the redisSortObject structure:
Typedef struct _redisSortObject {// The value of the sorted key robj *obj; // double score is used when sorting numeric values; // Sort string values with the BY option using robj *cmpobj; } u; } redisSortObject;Copy the code1) Create an array with the same length as the numbers list, each item of which is a redis. H /redisSortObject structure, as shown in the figure
2) Iterate over the number group and point the OBJ pointer of each array item to each item in the numbers list to form the one-to-one relationship between the OBJ pointer and the list item, as shown in Figure 21-2.
3) iterate through the array, convert the list items pointed to by each obj pointer to a floating point number of type double, and save this floating point number in the u.score property of the corresponding array items, as shown in the figure.
Implementation of the ALPHA option
By using the ALPHA option, the SORT command sorts keys that contain string values, sorting the keys alphabetically.
Implementation of the ASC option and DESC option
By default, the SORT command performs an ascending ASC SORT, with the results sorted from smallest to largest in value size. ASC ascending sort and DESC descending sort are both performed by the same quicksort algorithm, the differences between them are as follows: · When ascending sort is performed, the comparison function used by the sort algorithm produces ascending comparison results. · When descending sort is performed, the comparison function used by the sorting algorithm produces descending comparison results.
Implementation of the BY option
BY using the BY option, the SORT command can SORT a key BY specifying some string key, or some field that a hash key contains, as element weights. The following example uses the price of apple, banana, and cherry to sort the set key fruits and find the weight key according to the mode *-price given BY. For the “apple” element, the finder returns the weight key “apple-price”.
Redis > MSET apple-price 8 banana-price 5.5 cherry-price 7 OK redis> SORT fruits BY *-price 1)"banana"
2) "cherry"
3) "Apple"Copy the codeWith the LIMIT option, we can have the SORT command return only a portion of the sorted elements. The LIMIT option is of the format LIMIT: · The offset parameter indicates the number of sorted elements to skip. · The count parameter represents the number of sorted elements to return after skipping a given number of sorted elements.
Implementation of the GET option
By using the GET option, we can have the SORT command, after sorting the keys, find and return the values of the other keys based on the element being sorted and the pattern specified by the GET option. Implementation of #####STORE option By default, the SORT command only returns the SORT result to the client, but does not save the SORT result:
redis> SADD students "peter" "jack" "tom"
(integer) 3
redis> SORT students ALPHA
1) "jack"
2) "peter"
3) "tom"
Copy the codeHowever, by using the STORE option, we can STORE the sort result in the specified key and reuse it if necessary. SORT: At this stage, the command sorts the input keys using ALPHA, ASC, or DESC, BY, and results in a sorted result set. 2) LIMIT the size of the sort result set: In this step, the command uses the LIMIT option to LIMIT the size of the sort result set. Only those elements specified by the LIMIT option will be kept in the sort result set. 3) GET foreign keys: In this step, the command uses the GET option to find and GET the values of the specified keys based on the elements in the sort result set and the pattern specified by the GET option, and uses these values as a new sort result set. 4) Save the sorted result set: in this step, the command will use the STORE option to save the sorted result set to the specified key. 5) Return the sorted result set to the client: In the last step, the command iterates through the sorted result set and returns the elements of the sorted result set to the client in turn.
Chapter 22. Binary arrays
A bit array is an array, and the element of the array is a binary bit. In Redis, you can use a string object to represent a binary bit array. The string object is an SDS structure.
Implementation of the SETBIT command
Similarly, SETBIT is used to set the value of the bits in the bitarray at offset to value and return the old value to the client before the bits were set. ##### implementation of the BITCOUNT command The BITCOUNT command is used to count the number of bits 1 in the location array. The following describes the algorithms that can be used to implement the BITCOUNT command: 1. The traversal algorithm, which traverses each bit and counts the number of 1s, is O(N) 2. Table lookup algorithm, because 8 bits of binary 1 and 0 permutation is limited, can be carried out to build a table, will be calculated in all cases. And then get 8 bits at a time, and then lookup table comparison, get the number of 1 in these 8 bits. The time complexity is O(N/8). An 8-bit table with key length is shown, but takes up some memory space to store the table.
Implementation of BITOP command
Because the C language support directly to perform logic AND bytes (&), OR (|), logic XOR logic (^) AND logic (~) operation, so the BITOP commands AND, OR, XOR AND NOT four operations are directly based on the logic operation. Such as:
BITOP AND result x y
Copy the codeUse the BITOP command to sum x and y and save the results on the result key.
Chapter 23 Slow Query logs
The Redis slow query log function records command requests whose execution time exceeds the specified time limit. You can use the logs generated by this function to monitor and optimize the query speed. The server is configured with two options related to slow query logging: · Slowlog-log-slower than option specifies how many microseconds (1 second equals 1 000 000 microseconds) command requests are logged for execution. · The slowlog-max-len option specifies the maximum number of slow query logs to be saved by the server. If full, the oldest slow query log is deleted.
Chapter 24 Monitor
By executing the MONITOR command, the client can turn itself into a MONITOR, receiving and printing information in real time about the command requests currently being processed by the server.
