1. What is Redis?
Redis is a high performance key-value database based on memory.
2. The characteristics of the Reids
Redis is essentially a key-value in-memory database, much like memcached. The entire database is loaded into memory for operation, and periodically flushed to hard disk asynchronously. Because it is a pure memory operation, Redis has excellent performance and can process more than 100,000 read and write operations per second, making it the fastest key-value DB known.
It’s not just performance that makes Redis great. One of Redis’s biggest attractions is the ability to store multiple data structures. In addition, the maximum limit for a single value is 1GB, unlike memcached, which can only store 1MB of data. For example, using his List to do FIFO bidirectional linked List, to achieve a lightweight high performance message queue service, with his Set can do high-performance tag system and so on. Redis can also set expire times for stored key-values, so it can be used as a more powerful version of memcached.
The main disadvantage of Redis is that the database capacity is limited by physical memory, so it cannot be used for high-performance read and write of massive data. Therefore, Redis is mainly suitable for high-performance operations and operations with small data volume.
3. What are the benefits of redis?
(1) Fast, because the data is stored in memory, similar to HashMap. The advantage of HashMap is that the time complexity of search and operation is O(1). (2) It supports rich data types, including String, list, set, sorted set and hash
1) Stringset/ get/decr incr/mget, etc.; Application scenario: String is the most commonly used data type. Common key/value storage can be classified into this type. String is stored in Redis as a String by default, and is referenced by redisObject. When incR, DECr, etc., is converted to a numeric value for calculation, the encoding field of redisObject is int. 2) Common Hash commands: hget/hset/hgetall Application scenario: We want to store a user information object data, including the user ID, name, age, and date of birth. Based on the user ID, we want to obtain the name, age, or date of birth of the user. Implementation: Redis Hash is actually a HashMap that stores values internally and provides an interface to access the Map members directly. As shown in the figure, Key is the user ID and value is a Map. The Map's key is the attribute name of the member and its value is the attribute value. In this way, data can be modified and accessed directly through the Key of the internal Map (Redis calls the Key of the internal Map field), that is, the corresponding attribute data can be manipulated by Key(user ID) + field(attribute tag). HashMap is currently implemented in two ways: When the HashMap has a small number of members, Redis will compact store it in a way similar to the one-dimensional array to save memory, instead of adopting the real HashMap structure. In this case, the encoding of the redisObject of the corresponding value is zipmap. When the number of members increases, it automatically converts to a real HashMap, in which case the encoding is HT.hash3) a List of common commands: lpush/rpush/lpop/rpop/lrange, etc.; Application scenario: There are many application scenarios of Redis List, which is also one of the most important data structures of Redis. For example, The list structure of Twitter followers and fans can be implemented. Implementation: Redis list is a two-way linked list, that can support reverse lookup and traversal, more convenient operation, but brought part of the extra memory overhead, Redis internal many implementations, including sending buffer queues are also using this data structure. 4) Set of commonly used commands: sadd/spop/smembers/sunion, etc.; Application scenario: RedissetThe function provided externally is similar to that provided by list. It is a list function, except thatsetIt automatically reloads, so when you need to store a list of data and you don't want duplicate data,setIs a good choice, andsetProvides to determine whether a member is in asetAn important interface within a collection, which is also not provided by a list; Implementation method:setThe internal implementation of HashMap is a HashMap whose value is always null, which is actually computedhashThe way to quickly shed weight, this is alsosetCan provide a reason to determine whether a member is in the collection. 5) Sorted Set common commands: zadd/zrange/zrem/zcard, etc. Application scenario: Redis sortedsetThe use of scenarios andsetSimilar, but the difference issetIt's not automatically sorted, it's sortedsetMembers can be sorted by the user providing an additional priority (score) parameter, and the order is inserted, i.e., automatic sorting. When you need an ordered and non-repetitive list of collections, choose sortedsetData structures, such as Twitter's public Timeline, can be stored by post time as score, so that retrieval is automatically sorted by time. Sorted by RedissetHashMap and SkipList are used internally to ensure the storage and order of data. HashMap stores the mapping of members to Score, while SkipList stores all members and sorts according to the score stored in HashMap. High search efficiency can be obtained by using the structure of SkipList. And the implementation is relatively simple.Copy the code(3) support transactions, operations are atomic, the so-called atomic is the change of data or all executed, or all not executed (4) rich features: can be used for caching, messages, by key to set expiration time, will be automatically deleted after expiration
4. What advantages does Redis have over memcached?
(1) All memcached values are simple strings, and Redis, as an alternative, supports richer data types (2) Redis is much faster than memcached and (3) Redis can persist its data
5. What are the differences between Memcache and Redis?
(1) Memecache stores all the data in memory. It will hang up after power failure. The data cannot exceed the size of memory. Redis is partially stored on hard disk to ensure data persistence. (2) Data support type Memcache data type support is relatively simple. Redis has complex data types. (3) Different underlying models are used. The underlying implementation modes between them and the application protocols for communication with clients are different. Redis directly built the VM mechanism itself, because normal system calls to system functions would waste a certain amount of time moving and requesting.
6. What scenarios does Redis apply to?
Redis is best for all data in-momory scenarios, such as:
(1) Session Cache
One of the most common scenarios for using Redis is session cache. The advantage of caching sessions with Redis over other stores, such as Memcached, is that Redis provides persistence.
(2) Full page Cache (FPC)
In addition to the basic session token, Redis also provides a very simple FPC platform. Returning to the consistency issue, even if the Redis instance is restarted, users will not see a drop in page loading speed due to disk persistence, which is a huge improvement, similar to PHP native FPC.
(3) Queue
One advantage of Reids in the memory storage engine world is that it provides list and set operations, which makes Redis a good message queue platform to use. The operation Redis uses as a queue is similar to the push/pop operation of a list in a native programming language such as Python.
If you do a quick Google search for “Redis Queues”, you will immediately find a number of open source projects designed to use Redis to build great back-end tools for queues. For example, Celery has a background that uses Redis as a broker, which you can view from here.
(4) leaderboards/counters
Redis does a great job of incrementing or decrementing numbers in memory. Set and Sorted Set also make it very easy to perform these operations. Redis just provides these two data structures. So, to get the top 10 users from the sorted set — we call it “user_scores” — we just need to do the following:
Of course, this assumes that you are doing an ascending order based on your users’ scores. If you want to return users and their scores, you need to do this:
ZRANGE user_scores 0 10 WITHSCORES
A good example is Agora Games, implemented in Ruby, whose leaderboards are stored in Redis, as you can see here.
Publish/subscribe
Last (but certainly not least) is Redis’s publish/subscribe capabilities. There are a lot of publish/subscribe usage scenarios.
7. Redis cache invalidation policy and primary key invalidation mechanism
As caching systems periodically clean up invalid data, a primary key invalidation and obsolescence strategy is required. In Redis, live keys are referred to as volatile. When the cache is created, a lifetime is set for a given key, and when the key expires (lifetime 0), it may be removed. 1. Some operations that affect the lifetime can be removed by using the DEL command to delete the entire key, or overwritten by the SET and GETSET commands. That is, after modifying the value of the key and overwriting the value of another key and value, The current data has a different lifetime. For example, executing INCR on a key, LPUSH on a list, or HSET on a hash table does not change the lifetime of the key itself. On the other hand, if RENAME is used to RENAME a key, the renamed key has the same lifetime as before. Another possibility of the RENAME command is to try to RENAME a key with a lifetime to another key with a lifetime, another_key. In this case, the old another_key (and its lifetime) is deleted, and the old key is renamed another_key. Therefore, the new another_key has the same lifetime as the original key. Run the PERSIST command to remove the lifetime of a key without deleting the key, so that the key becomes a persistent key again. You can run the EXPIRE command on a key that already has a lifetime. The specified lifetime will replace the old lifetime. The accuracy of the expiration time is controlled within 1ms. The time complexity of primary key expiration is O (1). The EXPIRE and TTL commands are used together. Returns 1 on success; If the key does not exist or cannot be set, 0 is returned. Maxmemory defaults to 0 and does not specify the maximum cache. If new data is added and the maximum memory is exceeded, Redis will crash, so it must be set. When redis memory data sets grow to a certain size, a data obsolescence strategy is implemented. Redis offers six data elimination strategies: Volatile – lRU: Discard the least recently used data from a set with an expiration date (server.db[I].expires). Volatile – TTL: selects expired data from a set (server.db[I].expires) to be discarded. Volatile -random: Nullify any data selected from a set with an expiration date (server.db[I].expires). Allkeys-lru: Cull least recently used data from dataset (server.db[I].dict). Allkeys-random: random selection of data from a dataset (server.db[I].dict). No-enviction: Note that the six mechanisms used here are volatile and allKeys, which specify whether to eliminate data from a set with an expiration date or from the entire set. Lru, TTL, and RANDOM are the three different elimination strategies. Plus a no-enviction never recycle policy. Using policy rules: 1. If the data presents a power-law distribution, that is, some data have high access frequency and some data have low access frequency, then allKeys-LRU is used. 2. TTL and RANDOM are easier to understand and implement. The main reason is that Lru has used the elimination strategy least recently. The design will sort the keys according to the failure time, and then select the key that fails first for elimination
8. Why does Redis need to put all data in memory?
Redis reads data to memory for the fastest read/write speed and writes data to disk asynchronously. So Redis is characterized by fast and persistent data. If data is not kept in memory, disk I/O speed severely affects Redis performance. As memory gets cheaper and cheaper, Redis will become more and more popular.
If the maximum memory usage is set, new values cannot be inserted after the number of existing records reaches the memory limit.
Redis is single-process, single-thread
Redis uses queue technology to turn concurrent access into serial access, eliminating the overhead of traditional database serial control
10. How to solve the concurrency competition problem of Redis?
Redis is a single-process single-thread mode, which uses queue mode to change concurrent access into serial access. Redis itself does not have the concept of locking, Redis does not compete for multiple client connections, but when Jedis clients concurrently access Redis, connection timeout, data conversion error, blocking, client closing and other problems will occur, and these problems are all
The client connection is abnormal. Procedure There are two solutions to this:
1. From the client side, in order to ensure the normal and orderly communication between each client and Redis, the connection is pooled, and the client reads and writes Redis using internal synchronized.
2. Server Angle, using setNX to achieve lock. Note: For the first method, which requires the application to handle the resource synchronization itself, you can use a simple method, such as synchronized or lock. The second requires Redis’ setnx command, but there are a few caveats.
11. Redis Common performance Problems and solutions:
1). If the Master writes a snapshot, the save command will schedule the rdbSave function, which will block the work of the main thread. If the snapshot is large, the performance will be greatly affected, and the service will be interrupted intermittently.
2).Master AOF persistence. If you do not rewrite AOF files, this method has the least impact on performance, but AOF files will continue to grow, too large AOF files will affect the recovery speed of Master restart. It is best for the Master not to do any persistence work, including memory snapshots and AOF log files, and especially not to enable memory snapshots for persistence
If data is critical, a Slave enables AOF backup and synchronizes data once per second.
3).Master calls BGREWRITEAOF to rewrite the AOF file. AOF will occupy a large amount of CPU and memory resources during the rewrite, resulting in high load of services and temporary service suspension.
4). Redis is a performance problem of Master/Slave replication. For the speed of Master/Slave replication and the stability of connection, it is better for Slave and Master to be in the same LAN.
12. CAS(check-and-set operation for optimistic locking)?
Like many other databases, Redis as a NoSQL database also provides transaction mechanism. In Redis, the four commands MULTI/EXEC/DISCARD/WATCH are the cornerstones of our transaction implementation. This concept will be familiar to developers with relational database development experience, but we will list it briefly
The Redis
Transaction implementation features: 1). All commands in the transaction will be executed sequentially. During the execution of the transaction, Redis will not provide any service for requests from other clients, thus ensuring that all commands in the transaction will be executed atomically. 2). In a Redis transaction, if a command fails, subsequent commands will continue to be executed. 3). We can start a TRANSACTION with the MULTI command, which can be understood as a “BEGIN TRANSACTION” statement by those experienced in relational database development. Commands executed after this statement are considered operations within the transaction, and we can commit/roll back all operations within the transaction by executing EXEC/DISCARD commands. These two Redis commands can be considered equivalent to the COMMIT/ROLLBACK statements in a relational database.
4). Before the transaction starts, if the communication between the client and the server fails and the network is disconnected, all subsequent statements to be executed will not be executed by the server. However, if the network interruption event occurs after the EXEC command is executed by the client, all commands in the transaction will be executed by the server.
5). When the appends-only mode is used, Redis will call the system function write to write all the write operations in the transaction to the disk in this call. However, if a system crash occurs during the writing process, such as a power outage, then only part of the data may be written to disk and some of the data may be lost. The Redis server performs a series of necessary conformance checks upon restart and exits with an error message as soon as a similar problem is found. In this case, we need to take full advantage of the Redis toolkit provided with the Redis-check-aof tool, which can help us locate data inconsistency errors and roll back some of the data that has been written. After the repair we can restart the Redis server again.
13.WATCH command and Cas-based optimistic locking?
In Redis transactions, the WATCH command can be used to provide the check-and-set (CAS) function. Suppose we monitor multiple Keys through the WATCH command prior to the transaction execution. If any Key value changes after the WATCH, the EXEC command will abandon the transaction and return a Null multi-bulk reply to notify the caller of the transaction
The execution failed. Procedure For example, let’s again assume that Incr is not provided in Redis to do atomic increment of key values. We would have to write our own code to do this. The pseudo-code is as follows: val = GET mykey val = val + 1 SET mykey
14. Have you ever used Redis distributed lock? What is it?
Use setnx to fight for locks, and use expire to add an expiration time to locks in case they are forgotten to release.
You will be told that you answered well, and then asked what happens if the process crashes after setnx before expire or if maintenance needs to be restarted?
This is where you respond with surprise: Oh, yeah, the lock will never be released. The next thing you need to do is scratch your head, pretend to think about it for a moment, as if you were thinking about the next result on your own initiative, and reply: I remember that the set directive has very complicated parameters. It should be possible to combine setnx and EXPIRE into one directive. The other party will show a smile at this time, the heart began to recite: press, this boy is not bad.
15. Suppose there are 100 million keys in Redis, and 10W of them start with a fixed known prefix. What if you could find all of them?
Use the keys command to scan out a list of keys for a given pattern.
If the redis is providing services to online businesses, what is the problem with using keys?
This is the time to answer a key redis feature: Redis single threaded. The keys command causes the thread to block for a period of time and the online service to pause until the command is executed. In this case, scan command can be used. Scan command can extract the key list of the specified mode without blocking, but there will be a certain probability of repetition. It is ok to perform deduplication on the client, but the overall time will be longer than that of direct keys command.
16. Have you ever used Redis for asynchronous queues and how?
The list structure is typically used as a queue, with RPUSH producing messages and LPOP consuming messages. When there are no LPOP messages, sleep for a while and try again.
What if the other person asks if it’s ok not to sleep? List also has a directive called blPOP, which blocks when there is no message until it arrives.
If the other party asks can produce once consume many times? Using the PUB/SUB topic subscriber pattern, a 1:N message queue can be implemented.
What are the disadvantages of pub/sub? In the event that the consumer goes offline, the production message is lost and a professional message queue such as RabbitMQ is used.
If the other party asks how redis realizes the delay queue? I bet right now you’d want to beat the interviewer to death if you had a baseball bat in your hand. But you are very restrained and calmly answer: Use sortedSet, take timestamp as score, call Zadd for message content as key to produce message, and the consumer uses ZrangebyScore command to get data polling N seconds ago for processing.
At this point, the interviewer has secretly given you a thumbs-up. But what he doesn’t know is that right now you’re sticking your middle finger up behind your chair.
17. If a large number of keys need to be set to expire at the same time, what should I pay attention to?
If a large number of key expiration times are set too centrally, redis may experience a temporary lag at that point in time. It is generally necessary to add a random value to the time to spread out the expiration time.
18. How does Redis persist?
Bgsave does mirror full persistence and AOF does incremental persistence. Because BGSave takes a long time, is not real-time enough, and can result in massive data loss during downtime, AOF is needed to work with it. When a Redis instance is restarted, the memory is rebuilt using the BGSave persistence file and the aOF is used to replay the recent operation instructions to fully restore the pre-restart state.
What happens if all of a sudden the power goes out? Depending on the configuration of the SYNC property of the AOF log, if performance is not required, sync the disk with each write instruction and data will not be lost. However, it is unrealistic to sync every time due to high performance requirements. Sync is usually timed, such as 1S1 times, and at most 1s of data will be lost.
What is the principle of BGSave? You just give me two words, fork and cow. Cow refers to copy on write. After the child process is created, the parent process shares data segments. The parent process continues to provide read and write services, and the dirty page data is gradually separated from the child process.
19. What are the benefits of pipelines and why do you use them?
Multiple IO round trips can be reduced to one, provided that there is no causal correlation between the instructions executed by the pipeline. When using Redis-Benchmark for pressure measurement, it can be found that an important factor affecting the PEAK QPS of Redis is the number of pipeline batch instructions.
20. Do you know the synchronization mechanism of Redis?
Redis can use master slave synchronization and slave slave synchronization. During the first synchronization, the primary node performs a BGSave and records subsequent modifications to the memory buffer. After the synchronization is complete, the RDB file is fully synchronized to the replication node, and the replication node loads the RDB image to the memory. After the loading is complete, the master node is notified to synchronize the modified operation records to the replication node for replay.
21. Have you used Redis cluster? What is the principle of cluster?
Redis Sentinal focuses on high availability and automatically promotes slave to Master in the event of a master outage.
Redis Cluster focuses on scalability. When a single Redis memory is insufficient, a Cluster is used to fragment storage.
