1. Introduction to Redis
REmote DIctionary Server(Redis) is a key-value storage system written by Salvatore Sanfilippo. Redis is an open source, network-enabled, memory-based and persistent logging, key-value database written in ANSI C, BSD compliant, and provides multiple language apis. It is often called a data structure server because values can be of the types String, hash, list, set, and sorted sets.
Redis is a no SQL database based on key-value
1. Any binary sequence can be used as a key. 2. Redis has uniform rules for designing keysCopy the code2. Supported languages
ActionScript Bash C C# C++ Clojure Common Lisp
Crystal D Dart Elixir emacs lisp Erlang
Fancy gawk GNU Prolog Go Haskell Haxe Io Java Javascript
Julia Lua Matlab mruby Nim Node.js Objective-C
OCaml Pascal Perl PHP Pure Data Python R Racket
Rebol Ruby Rust Scala Scheme Smalltalk Swift Tcl VB VCL
Copy the code3. What are the application scenarios of Redis?
Session caches are the most commonly used ones. Message queues, such as payment, activity leaderboards or counts, publish and subscribe messages (message notifications), product lists, comment lists, etcCopy the code4. Redis installation
The general installation steps are as follows:
Redis is developed in C language, and the c language compilation environment is required to install Redis
Yum install gcc-C ++
Step 1: get the source code package: wget http://download.redis.io/releases/redis-3.0.0.tar.gz the second step: unzip redis: tar ZXVF redis - 3.0.0. Tar. The third step: gz compilation. Go to the redis source directory (CD Redis-3.0.0). Perform make step 4: Install. The make install PREFIX=/usr/local/redis #PREFIX parameter specifies the installation directory of redisCopy the code5. Redis data type
Redis supports five data types
3, sorted 3, sorted 4, sorted 5, sorted 5, sorted 5, sorted 5, sorted 5, sorted 5, sorted 5, sortedCopy the codeString is the most basic data type of redis. Each key corresponds to a value. Note that the maximum storage capacity of a key value is 512MB.
127.0.0.1:6379> set key "hello world"
OK
127.0.0.1:6379> get key
"hello world"
127.0.0.1:6379> getset key "nihao"
"hello world"
127.0.0.1:6379> mset key1 "hi" key2 "nihao" key3 "hello"
OK
127.0.0.1:6379> get key1
"hi"
127.0.0.1:6379> get key2
"nihao"
127.0.0.1:6379> get key3
"hello"
Copy the codeRelated commands
Getset Sets a value for a Key and returns the value for a Key. Mset Sets a value for multiple keys.Copy the codeA redis hash is a collection of key-value pairs. It is a mapping table of fields and values of string type. It is suitable for storing objects
127.0.0.1:6379> hset redishash 1" 001" (integer) 1 127.0.0.1:6379> hget redishash 1" 001" 127.0.0.1:6379> hmset Redishash 1" 001" 2" 002" OK 127.0.0.1:6379> hget redishash 1" 001" 127.0.0.1:6379> hget redishash 2" 002" 127.0.0.1:6379> hmget redishash 12 1) "001" 2) "002"Copy the codeRelated commands
Hset sets the field in the hash corresponding to the Key to value. If the hash does not exist, it is automatically created. Hget Obtains the field configuration value of a hash hmset Configures multiple field values of the same hash in batches hmGET obtains multiple field values of the same hash in batchesCopy the codeA list is redis’s simple list of strings, sorted by insertion order
127.0.0.1:6379> lpush word hi
(integer) 1
127.0.0.1:6379> lpush word hello
(integer) 2
127.0.0.1:6379> rpush word world
(integer) 3
127.0.0.1:6379> lrange word 0 2
1) "hello"
2) "hi"
3) "world"
127.0.0.1:6379> llen word
(integer) 3
Copy the codeRelated commands
Lpush Inserts elements to the left of the specified list and returns the length of the inserted list. Rpush inserts elements to the right of the specified list and returns the length of the inserted list. Llen Returns the length of the specified listCopy the codeA set is an unordered set of string and cannot be repeated
127.0.0.1:6379> sadd redis redisset
(integer) 1
127.0.0.1:6379> sadd redis redisset1
(integer) 1
127.0.0.1:6379> sadd redis redisset2
(integer) 1
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset"
3) "redisset2"
127.0.0.1:6379> sadd redis redisset2
(integer) 0
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset"
3) "redisset2"
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset3"
3) "redisset"
4) "redisset2"
127.0.0.1:6379> srem redis redisset
(integer) 1
127.0.0.1:6379> smembers redis
1) "redisset1"
2) "redisset3"
3) "redisset2"
Copy the codeRelated commands
Sadd adds a string element to the set corresponding to the key, and returns 1 on success or 0 if the element exists. Smembers returns all elements in the set specified. Srem removes an element from the set specifiedCopy the codeEach element in a sorted set needs to be assigned a score. The sorted elements are sorted in ascending order according to the score. If multiple elements have the same score, the sorted elements are sorted in ascending order in lexicographical order. The sorted set is therefore very good for ranking
127.0.0.1:6379> zadd nosql 0 001 (integer) 1 127.0.0.1:6379> zadd nosql 0 002 (integer) 1 127.0.0.1:6379> zadd nosql 0 003 (integer) 1 127.0.0.1:6379> zcount nosql 00 (integer) 3 127.0.0.1:6379> zcount nosql 03 (integer) 3 127.0.0.1:6379> zrem noSQL 002 (integer) 1 127.0.0.1:6379> zcount nosql 03 (integer) 2 127.0.0.1:6379> zscore nosql 003 "0" 127.0.0.1:6379> zrangebyScore nosql 0 10 1) "001" 2) "003" 127.0.0.1:6379> zadd noSQL 1 003 (integer) 0 127.0.0.1:6379> zadd nosql 1 004 (integer) 1 127.0.0.1:6379> zrangebyScore nosql 0 10 1) "001" 2) "003" 3) "004" 127.0.0.1:6379> zadd nosql 3 005 (integer) 1 127.0.0.1:6379> zadd nosql 2 006 (integer) 1 127.0.0.1:6379> zrangeByScore nosql 0 10 1) "001" 2) "003" 3) "004" 4) "006" 5) "005"Copy the codeRelated commands
Zadd Adds 1 or more elements to the specified sorteset zrem removes 1 or more elements from the specified sorteset Zcount Views the number of elements within the specified score range in the specified sorteset zScore views the elements in the specified score range in the specified sorteset Zrangebyscore views all elements within the specified score range in the specified sortesetCopy the code6, key-value related commands
127.0.0.1:6379> Exists key (integer) 1 127.0.0.1:6379> exists key1 (integer) 1 127.0.0.1:6379> Exists key100 (integer) 0 127.0.0.1:6379> get key "nihao,hello" 127.0.0.1:6379> get key1 "hi" 127.0.0.1:6379> del key1 (integer) 1 127.0.0.1:6379> get key1 (nil) 127.0.0.1:6379> Rename key key0 OK 127.0.0.1:6379> get key (nil) 127.0.0.1:6379> get key0 "Nihao,hello" 127.0.0.1:6379> type key0 string exists # check whether the key exists del # delete key expire # set key expiration time (in seconds) persist Rename # Rename Key type # Return value typeCopy the code7. Commands related to Redis service
127.0.0.1:6379> select 0 OK 127.0.0.1:6379> info # Server redis_version:3.0.6 redis_git_sha1:00000000 redis_git_dirty:0 Redis_build_id: 347e3EEEF5029F3 redis_mode:standalone OS :Linux 3.10.0-693.el7.x86_64 x86_64 ARCH_bits :64 Multiplexing_api: epoll gcc_version: 4.8.5 process_id: 31197 run_id: 8 b6ec6ad5035f5df0b94454e199511084ac6fb12 tcp_port: 6379 uptime_in_seconds:8514 uptime_in_days:0 hz:10 lru_clock:14015928 config_file:/usr/local/redis/redis.conf ------------------- omits line N 127.0.0.1:6379> CONFIG GET 0 (empty list or set) 127.0.0.1:6379> CONFIG GET 15 (empty list or) Set slect # flushdb # flushdb # delete the key from the currently selected database Flushall # flushes all database keysCopy the code8. Publishing and subscription of Redis
Redis publish and subscribe (PUB/SUB) is its message communication mode, with one side sending messages and one side receiving messages. Here are three clients subscribing to the same channel
The image below shows that when new information is sent to Channel 1, the message is sent to the three clients that subscribe to it
Redis transactions
Redis transactions can execute multiple commands at once
After receiving the exec command, execute the transaction operation. If a command fails to execute, other commands can continue to execute. 3Copy the codeA transaction goes through three phases: start transaction (multi) Command Execute end transaction (exec)
127.0.0.1:6379> MULTI OK 127.0.0.1:6379> Set key key1 QUEUED 127.0.0.1:6379> get key QUEUED 127.0.0.1:6379> Rename key QUEUED 127.0.0.1:6379> exec 1) OK 2) "key1" 3) OKCopy the code10. Redis security configuration
You can modify device password parameters in the configuration file to improve security
#requirepass foobared
Copy the codeIf no password is configured, the following query is performed
127.0.0.1:6379> CONFIG GET requirepass 1) "requirepass" 2)"Copy the codeAfter the password is configured, authentication is required
127.0.0.1:6379> CONFIG GET requirepass (error) NOAUTH Authentication required. 127.0.0.1:6379> AUTH foobared # Authentication OK 127.0.0.1:6379> CONFIG GET requirepass 1) "requirepass" 2) "foobared"Copy the code11. Redis persistence
Redis can persist in two ways :Snapshotting(snapshot), appends-only file(AOF)
Snapshotting (snapshot)
2. Save 900 1 #900 seconds if more than 1 Key is changed, start snapshot save 3. Save 300 10 #300 seconds if more than 10 keys are changed. 4. Save 60 10000 #60 seconds If more than 10000 keys are changed, start snapshot savingCopy the codeAppend-only file(AOF)
1. When using AOF persistence, Appendonly yes # enable appendfsync always of the AOF persistent storage Appendfsync Everysec # write to disk once per second. Appendfsync no # completely dependent on OSCopy the code12. Redis performance test
Come with related test tools
[root@test ~]# redis-benchmark --help Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests]> [-k <boolean>] -h <hostname> Server hostname (default 127.0.0.1) -p <port> Server port (default 6379) -s <socket> Server socket (overrides host and port) -a <password> Password for Redis Auth -c <clients> Number of parallel connections (default 50) -n <requests> Total number of requests (default 100000) -d <size> Data size of SET/GET value in bytes (default 2) -dbnum <db> SELECT the specified db number (default 0) -k <boolean> 1=keep alive 0=reconnect (default 1) -r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD Using this option the benchmark will expand the string __rand_int__ inside an argument with a 12 digits number in the specified range from 0 to keyspacelen-1. The substitution changes every time a command is executed. Default tests use this to hit random keys in the specified range. -P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline). -q Quiet. Just show query/sec values --csv Output in CSV format -l Loop. Run the tests forever -t <tests> Only run the comma separated list of tests. The test names are the same as the ones produced as output. -I Idle mode. Just open N idle connections and wait. Examples: Run the benchmark with the default configuration against 127.0.0.1:6379: $redis- Benchmark Use 20 PARALLEL requests, for a total of 100K requests, against 192.168.1.1: $redis-benchmark -h 192.168.1.1 -p 6379 -n 100000 -C 20 Fill 127.0.0.1:6379 with about 1 million keys only using the SET test: $redis-benchmark -t set -n 1000000 -r 100000000 benchmark 127.0.0.1:6379 for a few Commands Producing CSV output: $ redis-benchmark -t ping,set,get -n 100000 --csv Benchmark a specific command line: $ redis-benchmark -r 10000 -n 10000 eval 'return redis.call("ping")' 0 Fill a list with 10000 random elements: $ redis-benchmark -r 10000 -n 10000 lpush mylist __rand_int__ On user specified command lines __rand_int__ is replaced with a random integer with a range of values selected by the -r option.Copy the codeThe actual test performed 1 million requests simultaneously
[root@test ~]# redis-benchmark -n 1000000 -q PING_INLINE: 152578.58 requests per second PING_BULK: [root@test ~]# redis-benchmark -n 1000000 -q PING_INLINE: 152578.58 requests per second PING_BULK: 150308.14 Requests per second SET: 143266.47 Requests per second GET: 148632.58 requests per second INCR: 145857.64 Requests per second LPUSH: 143781.45 Requests per second LPOP: 147819.66 Requests per second SADD: 138350.86 Requests per second SPOP: 134282.27 Requests per second LPUSH (needed to Benchmark LRANGE): 141302.81 Requests per second LRANGE_100 (First 100 elements): 146756.67 Requests per second LRANGE_300 (First 300 elements): 148104.27 Requests per second LRANGE_500 (First 450 Elements): 152671.75 requests per second LRANGE_600 (First 600 Elements): 148104.27 requests per second MSET (10 keys): 132731.62 requests per secondCopy the code13. Backup and recovery of Redis
There are two methods of automatic backup in Redis: the first is to use dump. RDB file for backup and the second is to use aOF file for automatic backup
Dump. RDB Backup The default automatic file backup mode of the Redis service (if AOF is not enabled). When the service is started, data is automatically loaded from the dump. RDB file. Conf save 900 1 save 300 10 save 60 10000 You can also run the save command to manually back up data
127.0.0.1:6379> set name key
OK
127.0.0.1:6379> SAVE
OK
127.0.0.1:6379> set name key1
OK
127.0.0.1:6379> BGSAVE
Background saving started
Copy the codeWhen redis takes a snapshot for a dump file, the dump. RDB file is automatically generated
# The filename where to dump the DB
dbfilename dump.rdb
-rw-r--r-- 1 root root 253 Apr 17 20:17 dump.rdb
Copy the codeThe SAVE command is used to dump a snapshot of the database using the primary process. The BGSAVE command is used to fork a child process to perform snapshot backup. The difference between the two backups is that the former blocks the primary process and the latter does not.
To restore, for example,
# Note that you must specify a directory here, Not a file name. dir /usr/local/redisdata 127.0.0.1:6379> CONFIG GET dir 1) "dir" 2) "/usr/local/redisdata" 127.0.0.1:6379> Set key 001 OK 127.0.0.1:6379> Set KEY1 002 OK 127.0.0.1:6379> Set KEY2 003 OK 127.0.0.1:6379> Save OKCopy the codeBack up the backup file to another directory
[root@test ~]# ll /usr/local/redisdata/
total 4
-rw-r--r-- 1 root root 49 Apr 17 21:24 dump.rdb
[root@test ~]# date
Tue Apr 17 21:25:38 CST 2018
[root@test ~]# cp ./dump.rdb /tmp/
Copy the codeDelete the data
127.0.0.1:6379> del key1
(integer) 1
127.0.0.1:6379> get key1
(nil)
Copy the codeShut down the service and copy the original backup file back to the save backup directory
[root@test ~]# redis-cli -a foobared shutdown [root@test ~]# lsof -i :6379 [root@test ~]# cp /tmp/dump.rdb / usr/local/redisdata/cp: overwrite '/ usr/local/redisdata/dump RDB'? y [root@test ~]# redis-server /usr/local/redis/redis.conf & [1] 31487Copy the codeLog in to check whether the data is recovered
[root@test ~]# redis-cli -a foobared 127.0.0.1:6379> mget key1 key2 1) "001" 2) "002" 3) "003"Copy the codeAOF Automatic backup Redis is disabled by default
###### APPEND ONLY MODE ##########
appendonly no
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
# appendfsync always
appendfsync everysec
# appendfsync no
Copy the codeParameters related to the configuration file have been described in detail. AOF file backup is used to back up all historical records and executed commands. Similar to mysql binlog, AOF file backup is used to re-execute the commands executed before the restoration. It is important to note that before the restoration, the same as database restoration, you need to manually delete the del or command that has been incorrectly executed.
AOF backup is different from dump backup. AOF backup is different from dump backup. If only AOF is configured, AOF file recovery data is loaded upon restart. If both RBD and AOF are configured, only AOF file recovery data is loaded upon startup. If only RBD is configured, dump file recovery data is loaded upon startupCopy the codeNote: As long as aOF is configured but there is no AOF file, the database started at this time will be empty
14. Introduction to Redis production optimization
1. Memory management optimization
hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 When the #list members and values are not too large, they are stored in a compact format with relatively low memory overhead
When running Redis on Linux, if the system memory is small, automatic backup may fail. You need to change the vm. Overcommit_memory parameter, which is the memory allocation policy of the Linux system
0 means that the kernel checks to see if there is enough memory available for the process to use; If there is enough memory available, the memory request is allowed. Otherwise, memory requests fail and errors are returned to the application process. 1 means that the kernel allows all physical memory to be allocated, regardless of the current memory state. 2 indicates that the kernel is allowed to allocate more memory than all physical and swap space combinedCopy the codeRedis recommends changing the value of vm. Overcommit_memory to 1. You can do this in the following ways:
Overcommit_memory =1, /etc/sysctl.conf Overcommit_memory =1 (3) echo 1 > /proc/sys/vm-overcommit_memoryCopy the code2. Memory preallocation 3. Persistence mechanism
Periodic snapshot: inefficient and may cause data loss AOF: data integrity is maintained (the number of instances is not too large 2 gb maximum)Copy the codeOptimization summary 1) Select an appropriate data type based on service requirements 2) Disable all persistence methods when business scenarios do not require persistence (SSD disks are used to improve efficiency) 3) do not use virtual memory. 4) Do not use more than 3/5 of the total physical memory of the server where REDIS resides. 5) Use maxMemory. 6) Use multiple REDIS instances for large amounts of data
15. Redis cluster application
Clustering is the pooling of multiple Instances of Redis to meet the same business needs, or to achieve high availability and load balancing
What are the clustering solutions? 1, HaProxy + Keepalived + Redis cluster
1) Through the redis configuration file, to achieve master/slave replication, read/write separation 2) through the haProxy configuration, to achieve load balancing, when the slave failure will be timely removed from the cluster T 3) Keepalived to achieve high availability of load
2. Redis official Sentinel cluster management tool
Redis cluster production environment high availability solution actual combat process
1) Sentinel is responsible for monitoring, alerting and automatic failover of master/slave services in the cluster. 2) Redis cluster is responsible for providing external services
Redis Cluster Redis Cluster is a distributed solution of Redis, officially launched in Redis 3.0, effectively solve the needs of Redis distributed. Cluster architecture can be used to achieve load balancing when the bottleneck of single memory, concurrency, and traffic is encountered.
1) Official recommendation, no doubt about it. 2) Decentralized, the cluster can increase by 1000 nodes at most, and the performance expands linearly with the increase of nodes. 3) Convenient management, can add or remove nodes, move slots and so on. 4) Simple and easy to use.
