Solution 1 Customize yamL files to install Redis Cluster
background
Deploying a Redis cluster in Kubernetes is challenging because each Redis instance relies on a configuration file that keeps track of other cluster instances and their roles. To do this, you need to use a combination of Kubernetes StatefulSets and PersistentVolumes. Redis Cluster architecture schematic diagram:
Create the StatefulSet YAML file
--- apiVersion: v1 kind: ConfigMap metadata: name: redis-cluster data: update.sh: | #! /bin/sh REDIS_NODES="/data/nodes.conf" sed -i -e "/myself/ S / \ [0-9] {1, 3 \} \. \ [0-9] {1, 3 \} \. \ [0-9] {1, 3 \} \. \ [0-9] {1, 3 \} / ${POD_IP} / "${REDIS_NODES} exec" $@ "redis. Conf: | + bind 0.0.0.0 cluster - enabled yes cluster - the require - full - coverage no cluster - node - 30000 cluster timeout - the config file /data/nodes.conf cluster-migration-barrier 1 appendonly yes protected-mode no --- apiVersion: apps.kruise.io/v1beta1 # apiVersion: apps/v1 kind: StatefulSet metadata: name: redis-cluster spec: serviceName: redis-cluster replicas: 6 selector: matchLabels: app: redis-cluster template: metadata: labels: app: Redis-cluster spec: containers: -name: redis image: redis:6.2.1-alpine ports: - containerPort: 6379 name: client - containerPort: 16379 name: gossip command: ["/conf/update.sh", "redis-server", "/conf/redis.conf"] env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP volumeMounts: - name: conf mountPath: /conf readOnly: false - name: data mountPath: /data readOnly: false volumes: - name: conf configMap: name: redis-cluster defaultMode: 0755 volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 100Gi storageClassName: rbdCopy the codeNote:
Cluster-migration-barrier Primary nodes with a migration barrier will trigger a cluster allocation. Cluster-migration-barrier primary nodes with a migration barrier will trigger a cluster allocation. You are advised to keep the default value of protected-mode no in the production environment. If this parameter is enabled, only lookback IP (127.0.0.1) can be used to access Redis. An exception will be reported if the external network accesses Redis. The Advanced StatefulSet provided by kruise is used here. If kruise is not installed in the cluster, apps/v1 can be usedCopy the codeInstall redis cluster
[root@qd01-stop-k8s-master001 redis]# kubectl apply -f install-redis.yaml
configmap/redis-cluster created
statefulset.apps.kruise.io/redis-cluster created
[root@qd01-stop-k8s-master001 redis]# kubectl get po -n op
NAME READY STATUS RESTARTS AGE
redis-cluster-0 1/1 Running 0 3m26s
redis-cluster-1 1/1 Running 0 3m14s
redis-cluster-2 1/1 Running 0 2m54s
redis-cluster-3 1/1 Running 0 2m23s
redis-cluster-4 1/1 Running 0 2m14s
redis-cluster-5 1/1 Running 0 114s
Copy the codeCreate a redis – cluster service
---
apiVersion: v1
kind: Service
metadata:
name: redis-cluster
namespace: op
spec:
type: ClusterIP
ports:
- port: 6379
targetPort: 6379
name: client
- port: 16379
targetPort: 16379
name: gossip
selector:
app: redis-cluster
Copy the code[root@qd01-stop-k8s-master001 redis]# kubectl apply -f redis-svc.yml service/redis-cluster created [root@qd01-stop-k8s-master001 redis]# kubectl get svc -n op NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE redis-cluster ClusterIP 10.97.197.224 < None > 6379/TCP,16379/TCP 9s test can connect [root@qd01-stop-k8s-master001 redis]# Telnet 10.97.197.224 6379 Trying 10.97.197.224... Connected to 10.97.197.224. Escape character is '^]'.Copy the codeInitialize the redis cluster
Run the following command to obtain pod IP, and then use redis-cli –cluster to create a cluster
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $(kubectl -n op get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 {end}') >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 100.88.43.67:6379 to 100.64.147.152:6379 Adding Replica 100.113.170.5:6379 to 100.98.174.217:6379 Adding Replica 100.64.147.153:6379 to 100.80.158.227:6379 M: B47b27a3dbddf3fc1370cbe14ae753f4fce20b04 100.64.147.152:6379 slots: [0-5460] (5461 slots) master M: 09543217 c903350e963fc4fdf4acb73f8a1b7f8b 100.98.174.217:6379 slots: [5461-10922] (5462 slots) master M: 5389 ace495b68eeac85370d6783648dff68f2fb6 100.80.158.227:6379 slots: [10923-16383] (5461 slots) master S: B1f39714c006ae55b12b18e6537303d7a00e1704 100.64.147.153:6379 replicates 5389 ace495b68eeac85370d6783648dff68f2fb6 S: 0113 f4668ec2f3ca2e9470c44bd5faab532b0936 100.88.43.67:6379 replicates b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 S: 09543217 c903350e963fc4fdf4acb73f8a1b7f8b e1e2f18ae66c79f1943390beabb59613abbad38a 100.113.170.5:6379 replicates Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join .. >>> Performing Cluster Check (using node 100.64.147.152:6379) M: B47b27a3dbddf3fc1370cbe14ae753f4fce20b04 100.64.147.152:6379 slots: [0-5460] (5461 slots) master 1 additional up (s) S: 0113 f4668ec2f3ca2e9470c44bd5faab532b0936 100.88.43.67:6379 slots: (0 slots) slave replicates b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 M: 09543217 c903350e963fc4fdf4acb73f8a1b7f8b 100.98.174.217:6379 slots: [5461-10922] (5462 slots) 1 additional master replica(s) M: 5389 ace495b68eeac85370d6783648dff68f2fb6 100.80.158.227:6379 slots: [10923-16383] (5461 slots) 1 additional master Up (s) s: e1e2f18ae66c79f1943390beabb59613abbad38a 100.113.170.5:6379 slots: (0 slots) slave replicates 09543217c903350e963fc4fdf4acb73f8a1b7f8b S: B1f39714c006ae55b12b18e6537303d7a00e1704 100.64.147.153:6379 slots: (0 slots) slave replicates 5389ace495b68eeac85370d6783648dff68f2fb6 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.Copy the codeVerifying Cluster Information
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-0 -- redis-cli cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:178 cluster_stats_messages_pong_sent:181 cluster_stats_messages_sent:359 cluster_stats_messages_ping_received:176 cluster_stats_messages_pong_received:178 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:359 [root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-0 -- redis-cli cluster nodes 0113 f4668ec2f3ca2e9470c44bd5faab532b0936 100.88.43.67:6379 @ 16379 slave b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 0 1615348311156 1 connected 09543217 c903350e963fc4fdf4acb73f8a1b7f8b 100.98.174.217:6379 @ 16379 master - 0 1615348314162 2 Connected to 5461-10922 b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 100.64.147.152:6379 @ 16379 myself, master 1615348312000-0 1 connected a scale of 0-5460 to 5389 ace495b68eeac85370d6783648dff68f2fb6 100.80.158.227:6379 @ 16379 master - 0 1615348312000 3 Connected to 10923-16383 e1e2f18ae66c79f1943390beabb59613abbad38a 100.113.170.5:6379 @ 16379 slave 09543217c903350e963fc4fdf4acb73f8a1b7f8b 0 1615348313160 2 connected b1f39714c006ae55b12b18e6537303d7a00e1704 100.64.147.153:6379 @ 16379 slave ace495b68eeac85370d6783648dff68f2fb6 5389 0 1615348312158 3 connectedCopy the codeAs you can see from the output, the cluster has a total of six nodes, three master nodes and three slave nodes
Solution 2: Use kubeDB to install Redis
Install kubeDB
1, install KubeDB get AppsCode License se-issuer.appscode.com/ download KubeDB charts github.com/appscode/ch…
[root@qd01-stop-k8s-master001 kubedb-community]# unzip kubedb-community-v0.16.2.tgz
[root@qd01-stop-k8s-master001 kubedb-community]# cd kubedb-community
[root@qd01-stop-k8s-master001 kubedb-community]# ls -al
total 96
drwxr-xr-x 4 root root 158 Mar 10 15:26 .
drwxr-xr-x 3 root root 66 Mar 10 15:24 ..
-rw-r--r-- 1 root root 351 Feb 16 09:55 Chart.yaml
drwxr-xr-x 2 root root 28 Mar 10 15:24 ci
-rw-r--r-- 1 root root 493 Feb 16 09:55 doc.yaml
-rw-r--r-- 1 root root 353 Feb 16 09:55 .helmignore
-rw-r--r-- 1 root root 24422 Feb 16 09:55 README.md
drwxr-xr-x 2 root root 4096 Mar 10 15:24 templates
-rw-r--r-- 1 root root 47437 Feb 16 09:55 values.openapiv3_schema.yaml
-rw-r--r-- 1 root root 5230 Feb 16 09:55 values.yaml
Copy the codeYaml and put the License file in kubedb-community. 2. Install the License file using helm
[root@qd01-stop-k8s-master001 kubedb-community]# helm install kubedb-community --namespace kube-system --set-file license=./kubedb-community-license.txt -f values.yaml . NAME: kubedb-community LAST DEPLOYED: Wed Mar 10 15:38:59 2021 NAMESPACE: kube-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: To verify that KubeDB has started, run: kubectl get deployment --namespace kube-system -l "app.kubernetes.io/name=kubedb-community,app.kubernetes.io/instance=kubedb-community" Now install/upgrade appscode/kubedb-catalog chart. To install, run: Helm install kubedb-catalog appscode/kubedb-catalog --version v0.16.2 -- Namespace kube-system To upgrade, run: Helm upgrade kubedb-catalog appscode/kubedb-catalog --version v0.16.2 --namespace kube-system Run the following command to check whether the installation is complete [root@qd01-stop-k8s-master001 kubedb-community]# kubectl get deployment --namespace kube-system -l "app.kubernetes.io/name=kubedb-community,app.kubernetes.io/instance=kubedb-community" NAME READY UP-TO-DATE AVAILABLE AGE kubedb-community 1/1 1 38s Wait for CRDS registration success [root@qd01-stop-k8s-master001 kubedb-community]# kubectl get crds-l app.kubernetes.io/name=kubedb -w NAME CREATED AT elasticsearches.kubedb.com 2021-03-10T07:39:42Z elasticsearchversions.catalog.kubedb.com 2021-03-10T07:39:45Z etcds.kubedb.com 2021-03-10T07:39:42Z etcdversions.catalog.kubedb.com 2021-03-10T07:39:45Z memcacheds.kubedb.com 2021-03-10T07:39:43Z memcachedversions.catalog.kubedb.com 2021-03-10T07:39:45Z mongodbs.kubedb.com 2021-03-10T07:39:43Z mongodbversions.catalog.kubedb.com 2021-03-10T07:39:45Z mysqls.kubedb.com 2021-03-10T07:39:43Z mysqlversions.catalog.kubedb.com 2021-03-10T07:39:46Z perconaxtradbs.kubedb.com 2021-03-10T07:39:43Z perconaxtradbversions.catalog.kubedb.com 2021-03-10T07:39:46Z pgbouncers.kubedb.com 2021-03-10T07:39:44Z pgbouncerversions.catalog.kubedb.com 2021-03-10T07:39:46Z postgreses.kubedb.com 2021-03-10T07:39:44Z postgresversions.catalog.kubedb.com 2021-03-10T07:39:46Z proxysqls.kubedb.com 2021-03-10T07:39:44Z proxysqlversions.catalog.kubedb.com 2021-03-10T07:39:46Z redises.kubedb.com 2021-03-10T07:39:45Z redisversions.catalog.kubedb.com 2021-03-10T07:39:46ZCopy the code3, Install KubeDB Catalog again, download github.com/appscode/ch…
[root@qd01-stop-k8s-master001 kubedb-catalog]# tar -zxf kubedb-catalog-v0.16.2.tgz
[root@qd01-stop-k8s-master001 kubedb-catalog]# cd kubedb-catalog
[root@qd01-stop-k8s-master001 kubedb-catalog]# ls -al
total 24
drwxr-xr-x 3 root root 148 Mar 10 15:48 .
drwxr-xr-x 3 root root 28 Mar 10 15:48 ..
-rw-r--r-- 1 root root 321 Jan 26 20:08 Chart.yaml
-rw-r--r-- 1 root root 467 Jan 26 20:08 doc.yaml
-rw-r--r-- 1 root root 353 Jan 26 20:08 .helmignore
-rw-r--r-- 1 root root 3195 Jan 26 20:08 README.md
drwxr-xr-x 12 root root 188 Mar 10 15:48 templates
-rw-r--r-- 1 root root 744 Jan 26 20:08 values.openapiv3_schema.yaml
-rw-r--r-- 1 root root 1070 Jan 26 20:08 values.yaml
[root@qd01-stop-k8s-master001 kubedb-catalog]# helm install kubedb-catalog --namespace kube-system -f values.yaml .
NAME: kubedb-catalog
LAST DEPLOYED: Wed Mar 10 15:50:50 2021
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
Copy the codeInstall Redis using Kubedb
1. Take a look at the official redis life cycle diagram
Features Availability Clustering ✓ Instant Backup university Scheduled Backup university Persistent Volume ✓ Initialize using Snapshot university Initialize Using Script qualify Custom Configuration ✓ Using Custom docker image ✓ Builtin Prometheus Discovery ✓ using Prometheus operator ✓Copy the code2. View the supported versions
[root@qd01-stop-k8s-master001 kubedb-catalog]# kubectl get redisversions NAME VERSION DB_IMAGE DEPRECATED AGE 4.0.11 4.0.11 kubedb/redis:4.0.11 15m 4.0.6-v2 4.0.6 kubedb/redis:4.0.6-v2 15m 5.0.3-v1 5.0.3 kubedb/redis:5.0.3-v1 15m 6.0.6 kubedb/redis: 4.0.3-v1 15m 6.0.6 6.0.6 kubedb/redis: 6.0.6 15 mCopy the code3. To edit the yamL installation file, see github.com/kubedb/docs… Here choose to install version 6.0.6, my cluster storageClassName: “RBD”, please modify according to the actual situation if you want to customize redis. Conf, please refer to github.com/kubedb/docs…
ApiVersion: kubedb.com/v1alpha2 kind: Redis metadata: name: redis-cluster namespace: op spec: version: 6.0.6 mode: Cluster cluster: master: 3 replicas: 1 storageType: Durable storage: resources: requests: storage: 1Gi storageClassName: "rbd" accessModes: - ReadWriteOnceCopy the codePerform the installation
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl apply -f redis-cluster.yaml redis.kubedb.com/redis-cluster Created is installed, [root@qd01-stop-k8s-master001 kubedb-community]# kubectl get rd, po-n op NAME VERSION STATUS AGE redis.kubedb.com/redis-cluster 6.0.6 Provisioning 6m55s NAME READY STATUS RESTARTS AGE pod/ Redis-cluster-shard0-0 1/1 Running 0 6m54s pod/redis-cluster-shard0-1 1/1 Running 0 6m18s pod/redis-cluster-shard1-0 1/1 Running 0 5m38s pod/redis-cluster-shard1-1 1/1 Running 0 5m1s pod/redis-cluster-shard2-0 1/1 Running 0 4m30s pod/redis-cluster-shard2-1 1/1 Running 0 4m8s [root@qd01-stop-k8s-master001 redis]# kubectl get svc -n op NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE Redis-cluster ClusterIP 10.97.197.224 < None > 6379/TCP 5H16m Redis-cluster-Pods ClusterIP None < None > 6379/TCP 17MCopy the code4. Verify the cluster
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl get pods -n op -o jsonpath='{range.items[*]}{.metadata.name} -- -- -- -- -- -- -- -- -- -- {. Status. PodIP} : 6379 {" \ t \ n "}} {end '| grep redis redis - cluster - shard0-0 -- -- -- -- -- -- -- -- -- -- 100.64.147.156:6379 Redis - cluster - shard0-1 -- -- -- -- -- -- -- -- -- -- 100.98.174.218:6379 redis - cluster - shard1 0 -- -- -- -- -- -- -- -- -- -- 100.126.252.204:6379 Redis - cluster - shard1-1 -- -- -- -- -- -- -- -- -- -- 100.113.170.6:6379 redis - cluster - shard2 0 -- -- -- -- -- -- -- -- -- -- 100.107.55.69:6379 Redis-cluster-shard2-1 ---------- 100.78.230.4:6379 [root@qd01-stop-k8s-master001 redis]# kubectl-n op exec-it redis-cluster-shard0-0 -- redis-cli cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:864 cluster_stats_messages_pong_sent:882 cluster_stats_messages_sent:1746 cluster_stats_messages_ping_received:879 cluster_stats_messages_pong_received:864 cluster_stats_messages_meet_received:3 cluster_stats_messages_received:1746 [root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-shard0-0 -- redis-cli cluster nodes 1895cb4b9c31b848666c61000e502f55a29a8255 100.64.147.155:6379 @ 16379 master - 0 1615365162008 2 connected bdbf2ca37001774498a9b935afbc1cd2ce389c 30, 5461-10922 100.126.252.203:6379 @ 16379 slave 2 c06092fafa99e0158e39e6237a04fed25be3550 0 1615365163000 1 connected 9 b2cfbd5c1b417121d410141b6da9512ad29ce3c 100.78.230.3:6379 @ 16379 slave e83446c368839c5fdccf5f70e3b1004eb67cb651 0 1615365163512 3 connected 2 c06092fafa99e0158e39e6237a04fed25be3550 100.82.197.130:6379 @ 16379 myself, master - 0 1615365162000 1 connected a scale of 0-5460 to 1379 d2b20f26ab13d53068d276ec5d988b7a0273 100.64.122.197:6379 @ 16379 slave 1895cb4b9c31b848666c61000e502f55a29a8255 0 1615365163000 2 connected e83446c368839c5fdccf5f70e3b1004eb67cb651 100.107.55.68:6379@16379 master-0 1615365164014 3 Connected 10923-16383Copy the code