1. Introduction to Redis
REmote DIctionary Server(Redis) is a key-value storage system written by Salvatore Sanfilippo. Redis is an open source, network-enabled, memory-based and persistent logging, key-value database written in ANSI C, BSD compliant, and provides multiple language apis. It is often called a data structure server because values can be of the types String, hash, list, set, and sorted sets. Redis is a no SQL database based on key-value
1. Any binary sequence can be used as a key. 2. Redis has uniform rules for designing keys
2. Supported languages:
ActionScript Bash C C# C++ Clojure Common LispCrystal D Dart Elixir emacs lisp Erlang Fancy gawk GNU Prolog Go Haskell Haxe Io Java Javascript Julia Lua Matlab mruby Nim Node.js Objective-C OCaml Pascal Perl PHP Pure Data Python R Racket Rebol Ruby Rust Scala Scheme Smalltalk Swift Tcl VB VCL*Copy the code3. What are the application scenarios of Redis?
Session caches are the most commonly used ones. Message queues, such as payment, activity leaderboards or counts, publish and subscribe messages (message notifications), product lists, comment lists, etc
4. Redis installation
The general installation steps are as follows:
Yum install gcc-C ++ yum install gcc-C ++
Step 1: get the source code package: wget http://download.redis.io/releases/redis-3.0.0.tar.gz the second step: unzip redis: tar ZXVF redis – 3.0.0. Tar. The third step: gz compilation. Go to the redis source directory (CD Redis-3.0.0). Perform make step 4: Install. The make install PREFIX=/usr/local/redis #PREFIX parameter specifies the installation directory of redis
5. Redis data type
Redis supports five data types
3, sorted 3, sorted 4, sorted 5, sorted 5, sorted 5, sorted 5, sorted 5, sorted 5, sorted 5, sorted
String (string)
It is the most basic data type of redis. Each key corresponds to a value. Note that the maximum storage capacity of a key value is 512MB.
128.127.0.0.1:6379 >set key "hello world"OK 127.0.0.1:6379 > get the key"hello world"127.0.0.1:6379 > getset key"nihao"
"hello world"127.0.0.1:6379 > mset key1"hi" key2 "nihao" key3 "hello"OK 127.0.0.1:6379 > get key1"hi"127.0.0.1:6379 > get key2"nihao"127.0.0.1:6379 > get key3"hello"
Copy the codeRelated commands
Getset Sets a value for a Key and returns the value for a Key. Mset Sets a value for multiple keys.
Hash (hash)
Redis hash is a collection of key-value pairs. It is a mapping table of fields and values of string type. It is suitable for storing objects
128.127.0.0.1:6379 > hset redishash 1"001"
(integer) 1
127.0.0.1:6379> hget redishash 1
"001"127.0.0.1:6379 > hmset redishash 1"001" 2 "002"
OK
127.0.0.1:6379> hget redishash 1
"001"127.0.0.1:6379 > hget redishash 2"002"127.0.0.1:6379> hmget redishash 12 1)"001"
2) "002"
Copy the codeRelated commands
Hset sets the field in the hash corresponding to the Key to value. If the hash does not exist, it is automatically created. Hget Obtains the field configuration value of a hash hmset Configures multiple field values of the same hash in batches hmGET obtains multiple field values of the same hash in batches
The list (list)
Redis is a simple list of strings, sorted by insertion order
128.127.0.0.1:6379> lpush word hi
(integer) 1
127.0.0.1:6379> lpush word hello
(integer) 2
127.0.0.1:6379> rpush word world
(integer) 3
127.0.0.1:6379> lrange word 0 2
1) "hello"
2) "hi"
3) "world"127.0.0.1:6379 > llen word (integer) 3
Copy the codeRelated commands
Lpush Inserts elements to the left of the specified list and returns the length of the inserted list. Rpush inserts elements to the right of the specified list and returns the length of the inserted list. Llen Returns the length of the specified list
Set (set)
Is an unordered collection of type string and cannot be repeated
128.127.0.0.1:6379> sadd redis redisset
(integer) 1
127.0.0.1:6379> sadd redis redisset1
(integer) 1
127.0.0.1:6379> sadd redis redisset2
(integer) 1
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset"
3) "redisset2"
127.0.0.1:6379> sadd redis redisset2
(integer) 0
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset"
3) "redisset2"1) 127.0.0.1:6379 > smembers redis"redisset1"
2) "redisset3"
3) "redisset"
4) "redisset2"
127.0.0.1:6379> srem redis redisset
(integer) 1
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset3"
3) "redisset2"
Copy the codeRelated commands
Sadd adds a string element to the set corresponding to the key, and returns 1 on success or 0 if the element exists. Smembers returns all elements in the set specified. Srem removes an element from the set specified
Zset (sorted set)
Each element in a sorted set needs to be assigned a score. If multiple elements have the same score, the sorted set is sorted in ascending lexicographical order. Therefore, the sorted set is very good for ranking
128.127.0.0.1:6379> zadd nosql 0 001
(integer) 1
127.0.0.1:6379> zadd nosql 0 002
(integer) 1
127.0.0.1:6379> zadd nosql 0 003
(integer) 1
127.0.0.1:6379> zcount nosql 0 0
(integer) 3
127.0.0.1:6379> zcount nosql 0 3
(integer) 3
127.0.0.1:6379> zrem nosql 002
(integer) 1
127.0.0.1:6379> zcount nosql 0 3
(integer) 2
127.0.0.1:6379> zscore nosql 003
"0"
127.0.0.1:6379> zrangebyscore nosql 0 10
1) "001"
2) "003"
127.0.0.1:6379> zadd nosql 1 003
(integer) 0
127.0.0.1:6379> zadd nosql 1 004
(integer) 1
127.0.0.1:6379> zrangebyscore nosql 0 10
1) "001"
2) "003"
3) "004"
127.0.0.1:6379> zadd nosql 3 005
(integer) 1
127.0.0.1:6379> zadd nosql 2 006
(integer) 1
127.0.0.1:6379> zrangebyscore nosql 0 10
1) "001"
2) "003"
3) "004"
4) "006"
5) "005"
Copy the codeRelated commands
Zadd Adds 1 or more elements to the specified sorteset zrem removes 1 or more elements from the specified sorteset Zcount Views the number of elements within the specified score range in the specified sorteset zScore views the elements in the specified score range in the specified sorteset Zrangebyscore views all elements within the specified score range in the specified sorteset
6, key-value related commands
128.127.0.0.1:6379 > exists key (integer1 127.0.0.1:6379> exists key1 (integer1 127.0.0.1:6379> exists key100 (integer) 0
127.0.0.1:6379> get key
"nihao,hello"127.0.0.1:6379 > get key1"hi"127.0.0.1:6379 > del key1 (integer) 1 127.0.0.1:6379> get key1 (nil) 127.0.0.1:6379> Rename key key0 OK 127.0.0.1:6379> get key (nil) 127.0.0.1:6379> get key (nil) 127.0.0.1:6379> get key0"nihao,hello"127.0.0.1:6379 >type key0
string
Copy the codeExists Verifies whether a key exists del Deletes key expire sets key expiration time (in seconds) persist Removes the configuration of key expiration time rename Renames key type The type of the returned value
7. Commands related to Redis service
128.127.0.0.1:6379> select 0
OK
127.0.0.1:6379> info
# ServerRedis_version :3.0.6 redis_git_SHA1:00000000 redis_git_dirty:0 redis_build_ID: 347e3EEEF5029F3 reDIS_mode :standalone OS :Linux 3.10.0-693.el7.x86_64 x86_64 arch_bits:64 multiplexing_API :epoll gcc_version:4.8.5 process_id:31197 run_id:8b6ec6ad5035f5df0b94454e199511084ac6fb12 tcp_port:6379 uptime_in_seconds:8514 uptime_in_days:0 hz:10 lru_clock:14015928 config_file:/usr/local/ redis/redis. Conf -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- to omit N 127.0.0.1:6379 > CONFIG GET 0 (the empty list orset)
127.0.0.1:6379> CONFIG GET 15
(empty list or set)
Copy the codeSlect Select the database (database 0-15) quit Quit the connection info Obtain service information and statistics monitor real-time monitoring config get Obtain service configuration flushdb Delete the key flushall in the currently selected database Delete keys from all databases
8. Publishing and subscription of Redis
Redis publish and subscribe (PUB/SUB) is its message communication mode, with one side sending messages and one side receiving messages. Here are three clients subscribing to the same channel
Redis transactions
Redis transactions can be more than an executive command, send into the queue before the exec command cache, the end of the transaction, after receiving the exec command to perform the transaction, if a command execution fails, other commands can still perform, in the implementation process of a transaction, the other client requests submitted will not be inserted into the transaction execution in the list of commands.
A transaction goes through three phases
Start transaction (multi) Command execute end transaction (exec)
128.127.0.0.1:6379 > MULTI OK 127.0.0.1:6379 >setKey key1 QUEUED 127.0.0.1:6379> get key QUEUED 127.0.0.1:6379> Rename key key001 QUEUED 127.0.0.1:6379>exec
1) OK
2) "key1"
3) OK
Copy the code10. Redis security configuration
You can modify device password parameters in the configuration file to improve security
# requirepass foobared Remove the commentThe # number configures the password127.0.0.1:6379> CONFIG GET requirepass 1)"requirepass"
2) ""After the password is configured, 127.0.0.1:6379> CONFIG GET requirepass (error) NOAUTH Authentication required. 127.0.0.1:6379> AUTH foobared# certification
OK
127.0.0.1:6379> CONFIG GET requirepass
1) "requirepass"
2) "foobared"
Copy the code11. Redis persistence
Redis can persist in two ways :Snapshotting(snapshot), appends-only file(AOF)
Snapshotting (snapshot)
2. Save 900 1 #900 seconds if more than 1 Key is changed, start snapshot save 3. Save 300 10 #300 seconds if more than 10 keys are changed. 4. Save 60 10000 #60 seconds If more than 10000 keys are changed, start snapshot saving
Append-only file(AOF)
1. When using AOF persistence, Appendonly yes # enable appendfsync always of the AOF persistent storage Appendfsync Everysec # write to disk once per second. Appendfsync no # completely dependent on OS
12. Redis performance test
Come with related test tools
[root@test ~]# redis-benchmark --help
Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests]> [-k <boolean>]
-h <hostname> Server hostname (default 127.0.0.1)
-p <port> Server port (default 6379)
-s <socket> Server socket (overrides host and port)
-a <password> Password forRedis Auth -c <clients> Number of parallel connections (default 50) -n <requests> Total number of requests (default 100000).-d <size> Data size of SET/GET value in bytes (default 2)
-dbnum <db> SELECT the specified db number (default 0)
-k <boolean> 1=keep alive 0=reconnect (default 1)
-r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD
Using this option the benchmark will expand the string __rand_int__
inside an argument with a 12 digits number in the specified range
from 0 to keyspacelen-1. The substitution changes every time a command
is executed. Default tests use this to hit random keys in the
specified range.
-P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline).
-q Quiet. Just show query/sec values
--csv Output in CSV format
-l Loop. Run the tests forever
-t <tests> Only run the comma separated list of tests. The testnames are the same as the ones produced as output. -I Idle mode. Just open N idle connections and wait. Examples: Run the benchmark with the default configuration against 127.0.0.1:6379: $redis-benchmark Use 20 parallel clients,forA total of 100K requests, against 192.168.1.1: $redis-benchmark -h 192.168.1.1 -p 6379 -n 100000 -C 20 Fill 127.0.0.1:6379 with about 1 million keys only using the SETtest:
$ redis-benchmark -t set -n 1000000 -r 100000000
Benchmark 127.0.0.1:6379 for a few commands producing CSV output:
$ redis-benchmark -t ping,set,get -n 100000 --csv
Benchmark a specific command line:
$ redis-benchmark -r 10000 -n 10000 eval 'return redis.call("ping")' 0
Fill a list with 10000 random elements:
$ redis-benchmark -r 10000 -n 10000 lpush mylist __rand_int__
On user specified command lines __rand_int__ is replaced with a random integer
with a range of values selected by the -r option.
Copy the codeThe actual test performed 1 million requests simultaneously
[root@test ~]# redis-benchmark -n 1000000 -q
PING_INLINE: 152578.58 requests per second
PING_BULK: 150308.14 requests per second
SET: 143266.47 requests per second
GET: 148632.58 requests per second
INCR: 145857.64 requests per second
LPUSH: 143781.45 requests per second
LPOP: 147819.66 requests per second
SADD: 138350.86 requests per second
SPOP: 134282.27 requests per second
LPUSH (needed to benchmark LRANGE): 141302.81 requests per second
LRANGE_100 (first 100 elements): 146756.67 requests per second
LRANGE_300 (first 300 elements): 148104.27 requests per second
LRANGE_500 (first 450 elements): 152671.75 requests per second
LRANGE_600 (first 600 elements): 148104.27 requests per second
MSET (10 keys): 132731.62 requests per second
Copy the code13. Backup and recovery of Redis
Redis automatic backup comes in two ways
The first is to use dump. RDB file for backup and the second is to use aOF file for automatic backup
Dump. RDB backup
The default automatic file backup mode of the Redis service (if AOF is not enabled) automatically loads data from the dump. RDB file when the service is started.
Conf save 900 1 Save 300 10 save 60 10000Copy the codeYou can also run the save command to manually back up data
128.127.0.0.1:6379 >setName key OK 127.0.0.1:6379> SAVE OK 127.0.0.1:6379>setName key1 OK 127.0.0.1:6379> BGSAVE Background Saving Started Redis Automatically generates dump. RDB files when snapshots are added to dump files# The filename where to dump the DB
dbfilename dump.rdb
-rw-r--r-- 1 root root 253 Apr 17 20:17 dump.rdb
Copy the codeThe SAVE command is used to dump a snapshot of the database using the primary process. The BGSAVE command is used to fork a child process to perform snapshot backup. The difference between the two backups is that the former blocks the primary process and the latter does not.
To restore, for example,
# Note that you must specify a directory here, not a file name.
dir /usr/local/redisdata/ Backup file storage path 127.0.0.1:6379> CONFIG GET dir 1)"dir"
2) "/usr/local/redisdata"127.0.0.1:6379 >setThe key 001 OK 127.0.0.1:6379 >setKey1 002 OK 127.0.0.1:6379 >setKey2 003 OK 127.0.0.1:6379> Save OKCopy the codeBack up the backup file to another directory
[root@test ~]# ll /usr/local/redisdata/
total 4
-rw-r--r-- 1 root root 49 Apr 17 21:24 dump.rdb
[root@test ~]# date
Tue Apr 17 21:25:38 CST 2018
[root@test ~]# cp ./dump.rdb /tmp/
Copy the codeDelete the data
128.127.0.0.1:6379 > del key1 (integer) 1
127.0.0.1:6379> get key1
(nil)
Copy the codeShut down the service and copy the original backup file back to the save backup directory
[root@test ~]# redis-cli -a foobared shutdown
[root@test ~]# lsof -i :6379
[root@test ~]# cp /tmp/dump.rdb /usr/local/redisdata/Cp: overwrite '/ usr /local/ redisdata/dump. RDB '? y [root@test ~]# redis-server /usr/local/redis/redis.conf &[1], 31487Copy the codeLog in to check whether the data is recovered
[root@test ~]# redis-cli -a foobared
127.0.0.1:6379> mget key key1 key2
1) "001"
2) "002"
3) "003"
Copy the codeAOF automatic backup
The redis service disables this configuration by default
###### APPEND ONLY MODE ##########
appendonly no
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
# appendfsync always
appendfsync everysec
# appendfsync no
Copy the codeParameters related to the configuration file have been described in detail. AOF file backup is used to back up all historical records and executed commands. Similar to mysql binlog, AOF file backup is used to re-execute the commands executed before the restoration. It is important to note that before the restoration, the same as database restoration, you need to manually delete the del or command that has been incorrectly executed.
AOF is different from dump backup
Aof file backup is different from dump file backup. If only AOF is configured, AOF file recovery data is loaded upon restart. If both RBD and AOF are configured, only AOF file recovery data is loaded upon startup. If only RBD is configured, dump file recovery data is loaded upon startup
Note: As long as aOF is configured but there is no AOF file, the database started at this time will be empty
14. Introduction to Redis production optimization
1. Memory management optimization
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
Copy the codeIf the list members and values are not too large, the list will be stored in compact format, and the memory cost is relatively small. If the system memory is small when running Redis in Linux, automatic backup may fail, and you need to modify the ** VM. Overcommit_memory ** parameter of the system. This parameter is the memory allocation policy of the Linux system.
0 means that the kernel checks to see if there is enough memory available for the process to use; If there is enough memory available, the memory request is allowed. Otherwise, memory requests fail and errors are returned to the application process. 1 means that the kernel allows all physical memory to be allocated, regardless of the current memory state. 2 indicates that the kernel is allowed to allocate more memory than all physical and swap space combinedCopy the codeRedis recommends changing the value of vm. Overcommit_memory to 1. You can do this in the following ways: Overcommit_memory =1; /etc/sysctl.conf Overcommit_memory =1 (3) echo 1 > /proc/sys/vm-overcommit_memory
2. Pre-allocate memory
Persistence mechanism
Periodic snapshot: inefficient and may cause data loss AOF: data integrity is maintained (the number of instances is not too large 2 gb maximum)Copy the code
Optimize the summary
1) Select an appropriate data type based on service requirements. 2) Disable all persistence methods when business scenarios do not require persistence (SSD disks are used to improve efficiency). 3) Do not use virtual memory. 4) Do not use more than 3/5 of the total physical memory of the server where REDIS resides. 5) Use maxMemory. 6) Use multiple REDIS instances for large amounts of data
15. Redis cluster application
Clustering is the pooling of multiple Instances of Redis to meet the same business needs, or to achieve high availability and load balancing
What are the clustering solutions?
1, HaProxy + Keepalived + Redis cluster
1) Through the redis configuration file, to achieve master/slave replication, read/write separation 2) through the haProxy configuration, to achieve load balancing, when the slave failure will be timely removed from the cluster T 3) Keepalived to achieve high availability of load
2. Redis official Sentinel cluster management tool
3, Redis Cluster
Redis Cluster is a distributed solution of Redis, officially launched in Redis 3.0 version, effectively solve the needs of Redis distributed. Cluster architecture can be used to achieve load balancing when the bottleneck of single memory, concurrency, and traffic is encountered.
