Redis requires basic commands, data structures, Lua scripts, and distributed locking

takeaway

This article introduces the use of Redis in daily work, which involves the data structure of Redis, corresponding commands, persistent configuration and Lua script, as well as the implementation scheme of distributed lock based on Redis. These are the necessary basic knowledge when using Redis. It is recommended to save the following commands

General command

#View the number of keys in the current library
dbsize

#Clearing the current library
flushdb

#Empty all libraries
flushall

#View all keys in the current library
keys *

#Whether the current library has a specified key
exists key1

#View the value type of key
type key1

#Remove the key
del key1

#Set the expiration time of the specified key, in seconds
expire key1 10

#View the remaining expiration time of the key. -1 indicates that the key will never expire, and -2 indicates that the key has expired
ttl key1

#Monitoring the key
watch key1

#Cancel monitor key
unwatch key1
Copy the code

Data structures and commands

string

Is a key/value data structure. A key corresponds to a string value. The maximum value of a single value is 512 meters

This is the most common data structure

The command

#Set a key/value
set key1 value1

#Set the expiration time when you set the key/value
setex key1 10 value1

#Gets the value of key
get key1

#Sets the new value and returns the old value
getset key1 value1

#Set multiple keys/values at a time,.... Key3, value3, etc
mset key1 value1 key2 value2 ...

#Get the value of multiple keys at once
mget key1 key2

#Appends content to the value of the specified key
append key1 xxx

#Gets the length of the value
strlen key1

#It succeeds only if the key does not exist
setnx key1 value1

#It succeeds only if all keys do not exist
msetnx key1 value1 key2 value2 ...

#Increments the value of the specified key by 1
incr key1

#Minus 1
decr key1

#Add the specified value to the value of the specified key. In this case, add 2
incrby key1 2

#Subtracts the value of the specified key by the specified value, in this case by 2
decrby key1 2

#Get the specified range of characters in the value of the specified key. For example, if the value is abcdefg, take 1 to 2 to return BC, that is, the characters containing positions 1 and 2
getrange key1 1 2

#Set the value of the specified position, specify the start position, and then directly overwrite. In the following example, if the value is abcdefg and the value from the first position is cb, the result is acbdefg
setrange key1 1 cb
Copy the code

list

A bidirectional linked list, an unordered, repeatable collection, usually used as a queue

The command

#Add elements from the table header. Value2 is the new table header
lpush key1 value1 value2 ...

#Add elements from the tail of the table. Value2 is the new tail
rpush key1 value1 value2 ...

#Ejects elements from the table header
lpop key1

#Ejects elements from the end of the table
rpop key1

#Pop an element from the end of the key1 table and add it to the head of the key2 table
rpoplpush key1 key2

#View the elements in the range of the specified index from the table
lrange key1 0 2

#View the entire list
lrange key1 1 0 -1

#Gets the element in a linked list that specifies its index from left to right
lindex key1 1

#Gets the last element in the list
lindex key1 -1

#Gets the length of the linked list
llen key1

#Insert value2 before ValuE1 in the linked list
linsert key1 before value1 value2

#Insert value2 after value1 in the linked list
linsert key1 after value1 value2

#Removes an element with the value value1 from a linked list, from left to right
lrem key1 1 value1

#Removes an element with the value value1 from a linked list, from right to left
lrem key1 -1 value1

#Delete all elements with value value1 from the linked list
lrem key1 0 value1
Copy the code

set

An unordered, unrepeatable set, often used to exclude duplicate data and random draws

The command

#Adds elements to the collection. Repeating elements are skipped
sadd key1 value1 value2 ...

#Take all the elements of the set
smembers key1

#Determine whether an element exists in the set
sismember key1 value1

#Gets the number of elements in the collection
scard key1

#Removes the specified element from the collection
srem key1 value1 value2 ...

#Pops an element from the collection at random and deletes it
spop key1

#It randomly takes elements from the set, but it does not delete them, and the 1 after that indicates the number of elements it takes
srandmember key1 1

#Find the intersection of two sets
sinter key1 key2

#Find the union of two sets
sunion key1 key2

#Find the difference set between two sets
sdiff key1 key2
Copy the code

zset

An ordered unrepeatable set, often used as a leaderboard

The command

#If you add an element, the same value but different scores will override the score
zadd key1 score1 value1 score2 value2

#Get the number of elements
zcard key1

#Take all the elements, from the smallest to the largest
zrange key1 0 -1

#Take some of the elements, start from small to large
zrange key1 0 4

#Take all the elements, from the largest to the smallest
zrevrange key1 0 -1

#Take some of the elements, and start from the largest to the smallest
zrevrange key1 0 4

#Take the elements whose score is within the specified range, from smallest to largest, where min and Max are the ranges of score
zrangebyscore key1 min max withscores

#Fetch the elements whose score is in the specified range, from largest to smallest
zrevrangebyscore key1 max min withscores

#Increments the score of the element with the specified value, where 1 is how much increments each time, and can be negative
zincrby key1 1 value1

#Delete the specified element
zrem key1 value1

#Count the number of elements in the set whose score is within the range
zcount key1 min max

#Returns the rank of the specified value in the collection, from smallest to largest, starting with 0
zrank key1 value1

#Returns the rank of the specified value in the collection, from largest to smallest
zrevrank key1 value1
Copy the code

hash

Similar to Java Map<String, String>

The command

#Add a key-value pair
hset key1 field1 value1

#Access to the key value
hget key1 field1

#Set key and value pairs in batches
hmset key1 field1 value1 field2 value2 ...

#Check whether the key exists
hexists key1 field1

#Get all keys
hkeys key1

#Get all values
hvals key1

#The key is incremented, and the next one is how much each increment is, and it can be negative, and when it's negative, it's decremented
hincrby key1 field1 1

#Successful if the key does not exist
hsetnx key1 field1 value1

#Gets all key-value pairs, odd for keys and even for values
hgetall key1
Copy the code

bitmap

Bitmap sets the value of each bit in the unit of bit (either 0 or 1). According to actual application scenarios, space-saving algorithms can be designed, such as Bloom filter. In this paper, user check-in is taken as an example, and the user ID is 1, one key every year, and key= user ID_ year, such as 1_2021

A user whose ID=1 checks in on day 2021-01-01, which is the first day of 2021 (that is, day 0), can run the following command to save the check-in record

#Set the 0th bit value of the key 1_2021 to 1 to indicate that the 0th day check-in was successful
setbit 1_2021 0 1
Copy the code

If the user checks in on 2021-01-03, run the following command

#Set the second bit value of the 1_2021 key to 1 to indicate that the second day check-in was successful
setbit 1_2021 2 1
Copy the code

You can now query the check-in status of the user in 2021 by using the get command

get 1_2021
#Output \ xa0
Copy the code

The get command outputs 0xA0, which is hexadecimal. When converted to binary, it is 10100000. The 1 bit in binary means that day was checked in, so day 0 and day 2 were checked in

Determine whether the user at 2021-01-03 has checked in

getbit 1_2021 2
Copy the code

Count how many checkins the user has in 2021, which is actually count how many ones there are

bitcount 1_2021
#The output of 2
Copy the code

This is not easy to implement. In the command provided by Redis, the unit of the specified range is byte. For example, the number of times in January 2021 is calculated from the 0th byte to the third byte (0, 1, 2 and 3 bytes in total). That is, the day of 2021-02-01 is also counted as follows:

#Counts the number of bits that are 1 in bytes 0 through 3, including byte 3
bitcount 1_2021 0 3
Copy the code

In this case, either by month to set the key value, or a separate query 2021-02-01 this day is check-in, if the total number of check-in is reduced by 1

From the example above, you can see that storing in bits is very space-saving, with 8 bits representing 8 days of checkups. You can also use a bitmap to store all users’ check-ins in a day. In this case, the user ID is taken as the offset of the bit. If the user ID is large and exceeds the maximum range of the bitmap, the user ID can be sharded to different bitmaps

The geographical position

Add multiple positions (longitude and latitude) in the same key to calculate the distance between each position. You can also specify the center of the circle to find the position meeting the conditions according to the radius, which can realize the nearby XXX function

The command

#Add a location called company to key1 at latitude and longitude 116.404844 39.915378Geoadd KEY1 116.404844 39.915378 company
#Add a location called home to key1 with latitude and longitude 116.370924 39.930871Geoadd key1 116.370924 39.930871 HOME
#Queries the latitude and longitude of the specified location
geopos key1 company

#Example Query the latitude and longitude of multiple locations
geopos key1 company home

#Calculate the distance between the two positions in m
geodist key1 company home

#Calculates the distance between two positions in km
geodist key1 company home km

#Take the specified longitude and latitude as the center of the circle, query all positions within the specified radius, where 116.370924 and 39.930871 are the longitude and latitude of the center of the circle, 2000 m is the size of the radius (unit: m), withdist represents the distance between the output position conforming to the condition and the center of the circle. Withcoord means to output the latitude and longitude of the position that meets the condition, and ASC means to order by distance from smallest to largestGeoradius key1 116.370924 39.930871 2000 m withdist withcoord ASC
#Queries all positions within the specified radius with the specified position as the center of the circle. The returned result contains the center of the circle. Other optional parameters are the same as those of the previous georadius
georadiusbymember key1 home 4000 m
Copy the code

persistence

Most of the time redis is used as a cache, which can be recovered from lost data, but sometimes it is used to store hot data, or Nginx directly connects to Redis to store some important data (which is difficult to recover after loss), so the data in Redis needs to be persistent

Redis provides two persistence modes: RDB and AOF. RDB is a full backup of the current data (known as a snapshot), and AOF records all written commands in the append mode. Therefore, generally, AOF files are larger, which may lead to full disk space

RDB

Running the save and bgsave commands generates a snapshot of the current memory data in the. RDB file. The bgsave command is executed on another thread and therefore does not block the main thread

By default, RDB is enabled in Redis. Redis automatically stores RDB.

Save 300 1000 # Is backed up every 300 seconds if 1000 keys change save 30 10000 # is backed up every 30 seconds if 10000 keys changeCopy the code

The above parameters must be set according to the data written to Redis. Do not create RDB snapshots too often. This will affect redis performance

AOF

In general, RDB and AOF should be enabled at the same time. The following describes how to set the AOF of redis. Edit the redis.conf configuration file and modify the following configuration items

#Open AOF
appendonly yes

#AOF the file name
appendfilename "xxx.aof"

#The interval at which commands are flushed to disk. Everysec is once per second
appendfsync everysec

#Data is not flushed to disk during bgrewrite
no-appendfsync-on-rewrite no
Copy the code

Appendfsync means: In computer architecture, we know that reading and writing to disk is slow relative to memory, so the CPU writes data to the memory buffer at a scheduled or full time. Redis’s AOF appendfsync configuration sets how often to write to disk. One second is a safe setting. If a fault occurs, only the data within the last 1 second will be lost

Compare RDB to AOF

The RDB periodically takes snapshots of the data in the memory. This affects the performance of Redis. Therefore, do not take snapshots too often
How does an error occur between snapshots in the RDB
RDB recovers faster when rebooting Redis, unlike AOF, which needs to be executed command by command
AOF can be set to add write commands to the AOF file once per second, so that the data loss is minimum when a fault occurs
The AOF file contains the redis write commands, so you can open the file to modify and delete the commands that are not needed
The downside of AOF is that redis has to record every step of the write command, so the file is large and needs to be slimmed down in time

The lua script

introduce

Lua scripts are supported by default in Redis, and we often use Lua scripts instead of Redis transactions to solve oversold and undersold situations

Here are the lua script features of Redis:

Atomic operations, lua scripts are executed as a whole and are not interrupted by commands from other connections, thus replacing transactions
The lua script can be reused after being loaded
Reduce the overhead of network requests. Send a lua script to Redis at one time, and Redis returns the result after execution, without multiple requests

usage

Direct execution

Parameter name 1 Parameter name 2 Value 1 Value 2Copy the code

Example:

eval "return KEYS[1].. ARGV[1]" 1 key1 val1#The output
key1val1
Copy the code

In the lua script, KEYS and ARGV have fixed names and start with an index of 1. In the example above, the number of arguments is 1. The parameter name 1 is key1 and the value 1 is val1

Note: If the script has more than one parameter, the parameter names are written together and the parameter values are written together, such as key1 key2 val1 val2 instead of key1 val1 key2 val2

Load scripts

The purpose of loading a script is to reuse it. The script load command returns a hash value of SHA1, which can then be used to invoke the loaded script

script load "return KEYS[1].. ARGV[1]"#Assuming that return 3783 a90bf1f43b15a1e06c4e7664da956ed959d9
Copy the code

Call the script

#3783 a90bf1f43b15a1e06c4e7664da956ed959d9 is the script the results returned by the load
#1 is the number of parameters
#Key1 is parameter name 1
#Val1 is the parameter value 1
evalsha 3783a90bf1f43b15a1e06c4e7664da956ed959d9 1 key1 val1
Copy the code

Call the script by specifying the sha1 hash value returned by the load script, and specify the number of arguments and the name and value of the arguments

Check whether the script is loaded

script exists 3783a90bf1f43b15a1e06c4e7664da956ed959d9
Copy the code

Java uses the Redis Lua script

This example uses the RedisTemplate provided by Spring to first introduce dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
Copy the code

Define a RedisClient that encapsulates common commands

@Component
public class RedisClient {
    @Autowired
    @Qualifier("redisTemplate")
    private RedisTemplate redisTemplate;

    /** * Execute the script *@paramClazz Returns the result type *@paramScript Lua script *@paramKeys Specifies the lua script parameter name *@paramArgs Lua script value *@param <T>
     * @return* /
    public <T> T execute(Class<T> clazz, String script, List<Object> keys, Object... args) {
        DefaultRedisScript<T> redisScript = new DefaultRedisScript<>();
        redisScript.setResultType(clazz);
        redisScript.setScriptText(script);

        return(T) redisTemplate.execute(redisScript, keys, args); }}Copy the code

Test use, first in redis set two key out, respectively is test_key1 and test_key2, lua script is the following two key increment operation, respectively increment 1 and 2

String script = "redis.call(\"INCRBY\", KEYS[1], ARGV[1])\nredis.call(\"INCRBY\", KEYS[2], ARGV[2])";

ArrayList<Object> keys = Lists.newArrayList("test_key1"."test_key2");
Object execute = redisClient.execute(Object.class, script, keys, 1.2);
System.out.println(execute);
Copy the code

A distributed lock

Redis has a setnx command, which can be set only if the key does not exist, and is often used to implement distributed locks. However, it is not safe to use the setnx command to implement distributed locks. If thread 1 successfully executes setnx and expires after 10 seconds, thread 2 will fail to execute setnx. If thread 1 fails to clear the lock in time during execution, other threads can only wait 10 seconds to acquire the lock. If thread 1 does not finish execution within 10 seconds, because the lock has expired, leading to other threads to perform setnX successfully, there are two threads at the same time to get the lock, such a way of use is certainly not good, today introduced the third-party library Redisson, Redisson to help us fix the above two cases need to solve the problem

Introduce dependencies first

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson-spring-boot-starter</artifactId>
</dependency>
Copy the code

Define a utility class, easy to use

@Component
public class RedissonUtils {
    @Autowired
    private RedissonClient redissonClient;

    private static RedissonClient client;

    @PostConstruct
    private void init(a) {
        client = redissonClient;
    }

    /** * Lock * by specifying key@paramKey Specifies the key *@return* /
    public static RLock lock(String key) {
        RLock lock = client.getLock(key);
        lock.lock();
        return lock;
    }

    /** * Specifies the time * by specifying the key and locking@paramKey Specifies the key *@paramExpire Lock duration, in milliseconds@return* /
    public static RLock lock(String key, long expire) {
        RLock lock = client.getLock(key);
        lock.lock(expire, TimeUnit.MILLISECONDS);
        return lock;
    }

    /** * Specifies the time * by specifying the key and locking@paramKey Specifies the key *@paramExpire Lock time *@paramUnit Unit of locking time *@return* /
    public static RLock lock(String key, long expire, TimeUnit unit) {
        RLock lock = client.getLock(key);
        lock.lock(expire, unit);
        return lock;
    }

    /** * Specifies the time to attempt to lock the key. If the lock fails, wait for the specified time *@paramKey Specifies the key *@paramWait Wait time for locking *@paramExpire Lock time *@paramUnit Unit of locking time *@return* /
    public static RLock tryLock(String key, long wait, long expire, TimeUnit unit) {
        RLock lock = client.getLock(key);
        try {
            if (lock.tryLock(wait, expire, unit)) {
                return lock;
            } else {
                return null; }}catch (InterruptedException e) {
            return null; }}/** * Unlock * by specifying a key@param key
     */
    public static void unlock(String key) {
        RLock lock = client.getLock(key);
        lock.unlock();
    }

    /** * Unlock * by specifying the lock@param lock
     */
    public static void unlock(RLock lock) {
        if(lock ! =null) { lock.unlock(); }}}Copy the code

Acquiring a lock

RLock lock = RedissonUtils.tryLock(LockKey.XXX, 5L.10L, TimeUnit.SECONDS);
Assert.isNull(lock, "Failed to acquire lock");

/ / to lock

try {
    // The operation to be performed after the lock is obtained. }finally {
    RedissonUtils.unlock(lock);
}
Copy the code

Note that the lock must be released in the finally in case an exception occurs and the lock is not released

Finally, redis is not necessary to use a distributed lock. There is a fatal problem with redis. When the primary redis is not synchronized to the secondary redis, the primary Redis will die and the system will switch to the secondary Redis, while other threads can still get the lock. So there are two threads in the system that have the lock

Redis requires basic commands, data structures, Lua scripts, and distributed locking

takeaway

General command

Data structures and commands

string

The command

list

The command

set

The command

zset

The command

hash

The command

bitmap

The geographical position

The command

persistence

RDB

AOF

Compare RDB to AOF

The lua script

introduce

usage

Direct execution

Load scripts

Call the script

Check whether the script is loaded

Java uses the Redis Lua script

A distributed lock

Related Posts

CLinkedList- double linked list list class

Study Notes Series (4) — JVM runtime memory region

Small examples of Stream details most commonly used in JDK1.8 work