preface
Regarding the use of Redis cluster function in Node, I have not found a relatively complete and easy to understand article, so I have taken many detours in the process of development and debugging.
This paper will introduce in detail how to set up the redis cluster locally, how to use the cluster in the client side, and summarize and explain the problems and errors encountered in the process of setting up, so as to avoid detour in the next use and improve the development and work efficiency.
The background of using cluster is: in Redis singlecase mode, with the increase of users and visits, the QPS value rises sharply 👆🏻, a large number of IO operations lead to a certain moment full of CPU (100%), at any time there is a risk of downtime, at the same time through batch processing Redis and other ways are also temporary, unable to break through the bottleneck of server performance. Therefore, using a cluster solution or adding redis instances is imperative.
Noun explanation — cluster
Cluster generally refers to the server cluster, different from distributed system, is a lot of servers together for the same service, in the eyes of the client like only one server. Clusters can use multiple computers for parallel computing to obtain high computing speed, and can also use multiple computers to do backup, so that any machine down the whole system can still run normally. (Prior to ReDIS3.0, sentry mode was generally used, but sentry configuration was slightly more complex, and performance and high availability were mediocre.)
Redis cluster requirements
Because the voting fault tolerance mechanism requires that more than half of the nodes think that a node is hung only when it is hung, two nodes cannot form a cluster, so the Redis cluster needs at least three nodes.
To ensure the high availability of the cluster, each node needs to have secondary nodes (that is, backup nodes), so the Redis cluster needs at least 6 servers. (Three master, three slave, three storage, three take, high availability, backup)
Of course, we can’t use that many servers for local debugging, so we can simulate running 6 instances of Redis locally, and in fact the redis cluster setup in production is basically the same as here.
Set up a local Redis cluster in MAC environment
1. Download and install Redis
You can choose to install it from the official website, or you can use the named line to install it
Brew install redis brew install redis brew install redis brew install redisCopy the code2. Configure the cluster environment using Redis
The first step is to locate the redis configuration file
brew list redis# Check where Redis is installedCD/opt/homebrew/Cellar/redis / 6.2.4Enter the folder where the version number is locatedopen .# Open folder- with
Xcode.appOpen thehomebrew.mxcl.redis.plist, can be foundredis.confThe location is as follows:
Create six service profiles
CD /opt/homebrew/etc/(configuration file directory found in the previous step)
Mkdir -p redis/cluster/7000 mkdir -p redis/cluster/7001 mkdir -p redis/cluster/7002 mkdir -p redis/cluster/7002 mkdir -p redis/cluster/7001 mkdir -p redis/cluster/7002 mkdir -p redis/cluster/7003 mkdir -p redis/cluster/7004 mkdir -p redis/cluster/7005Copy the codeModifying a Configuration File
/ opt/homebrew/etc/redis. The conf directory of the configuration files don’t need to modify, as long as to copy it to create redis/cluster / 7000 directory, then modify, the steps are as follows
- Make a copy of the configuration file first
CD/opt/homebrew/etc / # into the configuration file directory cp redis. Conf redis/cluster / 7000/7000. The conf code redis/cluster / 7000/7000. The conf # Open the configuration file with an editor or vim to make the changesCopy the code- Into the
7000.confAfter, modify the following properties
# Redis port number (7000-7005) port 7000 # cluster-enabled yes # cluster configuration file configuration file path, Conf (each configuration file must be modified.) cluster-config-file nodes-7000.conf # Timeout period for communication between nodes cluster-nodes-timeout 5000 # Data persistence appendOnly YesCopy the code- Copy 70.conf to the directory of each Redis service
CD/opt/homebrew/etc/redis cluster # into the configuration file directory cp 7000/7000. 7001/7001 conf. Conf cp 7000/7000. 7002/7002 conf. Conf cp 7000/7000.conf 7003/7003.conf cp 7000/7000.conf 7004/7004.conf cp 7000/7000.conf 7005/7005.confCopy the code- To modify the
7001.conf-7005.confPort and cluster-config-file properties for each configuration file
Note: each configuration file must be configured with different port and cluster-config-file values (otherwise the cluster will not take effect).
Run the find /opt/ homebrew-name nodes-7000.conf command to find the directory of the configuration file
3. Start or stop the cluster service
Since we configured six services, it is impossible to start or stop them one by one. Shell scripts are needed to achieve this
Into the/opt/homebrew/etc/redis cluster directory, create a start. The sh and stop. Sh file
# start.sh file #! /bin/sh redis-server /opt/homebrew/etc/redis/cluster/7000/7000.conf & redis-server /opt/homebrew/etc/redis/cluster/7001/7001.conf & redis-server /opt/homebrew/etc/redis/cluster/7002/7002.conf & redis-server /opt/homebrew/etc/redis/cluster/7003/7003.conf & redis-server /opt/homebrew/etc/redis/cluster/7004/7004.conf & redis-server /opt/homebrew/etc/redis/cluster/7005/7005.conf & # stop.sh File #! /bin/sh redis-cli -p 7000 shutdown & redis-cli -p 7001 shutdown & redis-cli -p 7002 shutdown & redis-cli -p 7003 shutdown & redis-cli -p 7004 shutdown & redis-cli -p 7005 shutdown &Copy the codeRun./start.sh or./stop.sh to start or stop the service
Perform ps – ef | grep redis to view your redis service is started
Note: The first execution of./start.sh requires sudo chmod +x start.sh to grant execution permission
4. Related commands
Redis -cli -p 7000 # redis-server 7000/7000.conf # Redis-cli -p 7000 shutdown # redis-server chmod +x Redis -cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 cluster Nodes # Check cluster nodes (run on a client) cluster info Flushes all keys in the Redis server: flushdb Flushes all keys in the current repository: flushdbCopy the codeClients access the cluster using the Ioredis framework
Redis.Cluster provides automatic sharding on multiple Redis nodes. You can test the effect of the Cluster by using six Redis servers and starting Node redis.js locally. ioredis
// redis.js const Redis = require("ioredis"); Const cluster = new Redis.Cluster([{port: 7000, host: "127.0.0.1",}, {port: 7001, host: "127.0.0.1",},]); cluster.set("foo", "bar"); cluster.get("foo", (err, res) => { // res === 'bar' });Copy the codeUsing the Bull framework (Redis queue)
Import Queue from 'bull' const instance = new Queue('custom', {prefix: '{myprefix}', createClient(type) {// cluster cluster instance ibid return cluster}}) // Add data to redis queue (producer) instance.add('request', {... params }, { removeOnComplete: False}). Catch (e => {console.error(e)}) // Consumer callback instance.process('request', 5, async (job, Done)=> {console.log(' get data currently consumed: ', job.data) // await new Promise((resolve)=>resolve()) done()})Copy the codeThere are problems when using Bull framework to connect ioredis cluster: every time there is data push to redis queue, the corresponding callback function may trigger several times. At present, we cannot determine whether the problem is caused by the use or the framework itself (please leave a message if you know about it).
Alternative clustering: Multiple Redis instances can be used on the client side without data synchronization and migration, combined with math.random () to split data to one of the Redis, thus eliminating the hardware (CPU, etc.) bottleneck of a single instance.
Problem processing
- Error in connecting to Redis in Mac system?
Could not connect to Redis at 127.0.0.1:6379: Connection refused
Cause: The server is not started or fails to be started
Solution: Start the Redis server reference link
- Client startup, read/write error?
Error: ClusterAllFailedError: Failed to refresh slots cache.
Cause: The cluster-config-file attribute in the configuration file of each service is the same.
Handling: Change to a unique attribute value reference link 1 reference 2
- Failed to execute create primary/secondary Redis statement?
Execute statement: Redis -cli –cluster create –cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:127.0.0.1 7004:7005
Error message: [ERR] Node 127.0.0.1:7000 is not empty. Either the Node already knows other Nodes (check with CLUSTER Nodes) or contains some key in database 0
Cause: The data was not cleared or the cluster was reset when the create statement was executed
Processing: Clear data and reset the cluster, clear RDB and AOF files
See Clearing Redis Data
Take port 7000 as an example. 7001-7005 Repeat $redis-cli -p 7000 127.0.0.1:7000> flushall 127.0.0.1:7000> cluster reset 127.0.0.1:7000> exit # Find /opt/ homebrew-name dump. RDB # redis-cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005Copy the codeconclusion
The Redis cluster is very simple to use on the client side, whereas the configuration on the server side can be tedious.
The specific use of the client only do a simple description, use process to pay attention to redis data synchronization and migration and other issues.
Clustering can improve service capabilities, support master/slave replication, sentinel mode, read/write separation, and bisect server stress. However, the system does not have automatic fault tolerance and recovery functions. In case of downtime, some read and write requests fail, reducing the system availability. Analyze and select different solutions based on the service situation.
Author: tager related articles address: https://juejin.cn/user/4353721776234743/posts copyright owned by the author. Commercial reprint please contact the author for authorization, non-commercial reprint please indicate the source.