Nebula Graph: nebula-graph.com.cn/posts/how-t…

What is Kubernetes

Kubernetes is an open source application for managing containerized applications on multiple hosts in the cloud platform. The goal of Kubernetes is to make deployment containerized applications simple and efficient. Kubernetes provides a mechanism for application deployment, planning, updating, and maintenance.

Kubernetes defines a set of building blocks in its design architecture to provide a mechanism by which applications can be deployed, maintained, and extended. The components that make up Kubernetes are designed to be loosely coupled and extensible so that they can be used for a wide variety of workloads. Extensibility is largely provided by the Kubernetes API, which is used primarily as an internal component of the extension and as a container running on Kubernetes.

Kubernetes consists of the following core components:

  • etcdSaves the state of the entire cluster
  • kube-apiserverProvides a unique entry point for resource operations and provides mechanisms for authentication, authorization, access control, API registration, and discovery
  • kube-controller-managerResponsible for maintaining cluster status, such as fault detection, automatic expansion, rolling updates, etc
  • kube-schedulerResponsible for resource scheduling, scheduling Pod to the corresponding machine according to the scheduled scheduling policy
  • kubeletResponsible for maintaining the container lifecycle and managing the Volume and network
  • kube-proxyResponsible for providing intra-cluster Service discovery and load balancing for services

Kubernetes and databases

Database containerization is a hot topic these days, so what benefits can Kubernetes bring to the database?

  • Fault recovery: Kubernetes provides fault recovery. If a database application fails, Kubernetes can restart it automatically or migrate the database instance to another node in the cluster
  • Storage management: Kubernetes provides rich storage access solutions, database applications can transparently use different types of storage systems
  • Load balancing: Kubernetes Service provides load balancing to balance external access across different database instance copies
  • Horizontal scaling: Kubernetes scales the number of replicas based on the resource utilization of the current database cluster to increase resource utilization

Many databases, such as MySQL, MongoDB and TiDB, work well in Kubernetes clusters.

Nebula Graph in Kubernetes

Nebula Graph is a distributed, open source Graph database consisting of Graphd for Query Engine, Storaged for data storage, and Meted for metadata. In Kubernetes practice, it primarily brings the following benefits to the Nebula Graph database:

  • Kubernetes balances the load among the different copies of Nebula Graphd, Metad, and Storaged. Graphd, Metad, and Storaged can automatically discover each other via Kubernetes’ domain name service.
  • With the StorageClass, PVC and PV, the underlying storage details can be shielded. Regardless of whether local volumes or cloud disks are used, Kubernetes can be transparently interconnected.
  • With Kubernetes, a Nebula cluster can be successfully deployed in seconds, and Kubernetes can upgrade a Nebula cluster without being aware of it.
  • The Nebula cluster can recover itself with Kubernetes, and a single copy of Crash can be pulled up again by Kubernetes without the involvement of operations personnel.
  • Kubernetes can scale the cluster flexibly based on the resource utilization of the current Nebula cluster to improve cluster performance.

Let’s talk about the practical content.

Cluster deployment

Hardware and software requirements

The machine and operating system parameters involved in this deployment are listed here

  • Centos-7.6.1810 x86_64 for operating system
  • Vm Configuration
    • 4 CPU
    • 8 gb of memory
    • 50 g system disk
    • 50 GB data disk A
    • 50 GB data disk B
  • Kubernetes cluster version V1.14 +
  • Nebula is available in V2.0.0-RC1
  • Use the local PV as the data store
  • CoreDNS version 1.6.0 +

K8s cluster planning

The following is a list of clusters

Server IP Nebula instance role
192.168.0.1 master
192.168.0.2 graphd, metad-0, storaged-0 node
192.168.0.3 graphd, metad-1, storaged-1 node
192.168.0.4 graphd, metad-2, storaged-2 node

K8s Components to be deployed

  • Install the Helm3
  • Prepare the local disk and install the local volume plug-in
  • Installing the Nebula Cluster

Install the Helm3

Helm is the guarantor of the Kubernetes cluster. Using Helm can greatly reduce the threshold for deploying applications using Kubernetes. This article does not introduce Helm in detail. If you are interested, please read the Getting Started Guide to Helm by yourself.

Download and Install Helm

To install the Helm, run the following command on the terminal

$wget https://get.helm.sh/helm-v3.5.2-linux-amd64.tar.gz $tar -zxvf helm/helm-v3.5.2-linux-amd64.tgz $mv linux-amd64/helm /usr/bin/helmCopy the code

Viewing the Helm Version

Run the helm version command to view the helm version. The command output is as follows:

version.BuildInfo{Version:"v3.5.2", GitCommit:"167aac70832d3a384f65f9745335e9fb40169dc2", GitTreeState:"dirty", GoVersion:"go1.15.7"}
Copy the code

Setting the Local Disk

Do the following configuration on each machine

Creating a mount directory

$ sudo mkdir -p /mnt/disks
Copy the code

Formatting data disks

$ sudo mkfs.ext4 /dev/diskA 
$ sudo mkfs.ext4 /dev/diskB
Copy the code

Mount data disks

$ DISKA_UUID=$(blkid -s UUID -o value /dev/diskA) 
$ DISKB_UUID=$(blkid -s UUID -o value /dev/diskB) 
$ sudo mkdir /mnt/disks/$DISKA_UUID
$ sudo mkdir /mnt/disks/$DISKB_UUID
$ sudo mount -t ext4 /dev/diskA /mnt/disks/$DISKA_UUID
$ sudo mount -t ext4 /dev/diskB /mnt/disks/$DISKB_UUID

$ echo UUID=`sudo blkid -s UUID -o value /dev/diskA` /mnt/disks/$DISKA_UUID ext4 defaults 0 2 | sudo tee -a /etc/fstab
$ echo UUID=`sudo blkid -s UUID -o value /dev/diskB` /mnt/disks/$DISKB_UUID ext4 defaults 0 2 | sudo tee -a /etc/fstab
Copy the code

Deploy the local volume plug-in

$curl https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/archive/v2.4.0.zip $unzip v2.4.0. ZipCopy the code

Modify v2.4.0 / helm/provisioner/values. The yaml part classes:

  • willhostDir: /mnt/fast-diskstohostDir: /mnt/disks.# storageClass: truetostorageClass: true. Then execute:
$ helm install local-static-provisioner --namespace default sig-storage-local-static-provisioner/helm/provisioner
Run the local-static-provisioner command to view the local-static-provisioner deployment
$ helm list
NAME                    	NAMESPACE	REVISION	UPDATED                              	STATUS  	CHART            	APP VERSION
local-volume-provisioner	default  	1       	2021-02-10 11:06:34.3540341 +0800 CST	deployed	provisioner-2.4.0	2.4.0 
Copy the code

Deploying the Nebula Cluster

Download Nebula Helm Chart

# Download Nebula Chart$ helm repo add nebula-charts https://vesoft-inc.github.io/nebula-docker-compose $ helm pull nebula-charts/nebula $ tar - ZXVF nebula - v2.0.0. TGZCopy the code

Set the Kubernetes node

Below is the list of Kubernetes nodes where we need to set the scheduling label for the node node. You can tag 192.168.0.2, 192.168.0.3, 192.168.0.4 with the nebula: “Cloud” tag.

Server IP kubernetes roles nodeName
192.168.0.1 master 192.168.0.1
192.168.0.2 node 192.168.0.2
192.168.0.3 node 192.168.0.3
192.168.0.4 node 192.168.0.4

Specific operations are as follows:

$kubectl label node 192.168.0.2 nebula="cloud" --overwrite 
$ kubectl  label node 192.168.0.3 nebula="cloud" --overwrite
$ kubectl  label node 192.168.0.4 nebula="cloud" --overwrite
Copy the code

Adjust the default values of values

The Nebula Helm Chart catalog is as follows:

Nebula ├ ─ ─ Chart. Yaml ├ ─ ─ the README. Md ├ ─ ─ templates │ ├ ─ ─ configmap. Yaml │ ├ ─ ─ deployment. The yaml │ ├ ─ ─ _helpers. TPL │ ├ ─ ─ NOTES. TXT │ ├ ─ ─ PDB. Yaml │ ├ ─ ─ serviceaccount. Yaml │ ├ ─ ─ service. The yaml │ └ ─ ─ statefulset. Yaml └ ─ ─ values. The yaml 1 directory, 11 filesCopy the code

Can be modified according to the running environment demand charts/nebula/values. The default value in yaml

Install Nebula through helm

$ helm install nebula charts/nebula 
Check the deployment status
$ helm status nebula
NAME: nebula
LAST DEPLOYED: Fri Feb 19 12:58:16 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Nebula Graph Cluster installed!

1. Watch all containers come up.
  $ kubectl get pods --namespace=default -l app.kubernetes.io=nebula -w
View the Nebula deployment on the K8s cluster
$ kubectl get pods --namespace=default -l app.kubernetes.io=nebula
NAME                             READY   STATUS    RESTARTS   AGE
nebula-graphd-676cfcf797-4q7mk   1/1     Running   0          6m
nebula-graphd-676cfcf797-whwqp   1/1     Running   0          6m
nebula-graphd-676cfcf797-zn5l6   1/1     Running   0          6m
nebula-metad-0                   1/1     Running   0          6m
nebula-metad-1                   1/1     Running   0          6m
nebula-metad-2                   1/1     Running   0          6m
nebula-storaged-0                1/1     Running   0          6m
nebula-storaged-1                1/1     Running   0          6m
nebula-storaged-2                1/1     Running   0          6m

Copy the code

To access the Nebula Cluster:

$kubectl get service nebula-graphd NAME TYPE cluster-ip external-ip PORT(S) AGE nebula-graphd NodePort 10.105.47.116 <none> 9669:31646/TCP,19669:30554/TCP,19670:32386/TCP 22m# Test with nebula- Console
$ docker run --rm -ti --entrypoint=/bin/sh vesoft/nebula-console:v2-nightly
Connect to the Graphd component via NodePort/ $nebula- console-addr 192.168.0.4 -port 31646 -u root -p vesoft 2021/02/19 05:04:55 [INFO] Connection pool is Initialized successfully Welcome to Nebula Graph v2.0.0-rc1! (root@nebula) [(none)]> show hosts; +---------------------------------------------------------------+------+----------+--------------+---------------------- +------------------------+ | Host | Port | Status | Leader count | Leader distribution | Partition distribution | +---------------------------------------------------------------+------+----------+--------------+---------------------- +------------------------+ |"nebula-storaged-0.nebula-storaged.default.svc.cluster.local"9779 | |"ONLINE"| | 0"No valid partition" | "No valid partition"| +---------------------------------------------------------------+------+----------+--------------+---------------------- +------------------------+ |"nebula-storaged-1.nebula-storaged.default.svc.cluster.local"9779 | |"ONLINE"| | 0"No valid partition" | "No valid partition"| +---------------------------------------------------------------+------+----------+--------------+---------------------- +------------------------+ |"nebula-storaged-2.nebula-storaged.default.svc.cluster.local"9779 | |"ONLINE"| | 0"No valid partition" | "No valid partition"| +---------------------------------------------------------------+------+----------+--------------+---------------------- +------------------------+ |"Total"| | | 0 | | | +---------------------------------------------------------------+------+----------+--------------+---------------------- +------------------------+ Got 4 rows (time spent 2608/4258 us)Copy the code

FAQ

How to build a Kubernetes cluster?

To build a high availability of Kubernetes can refer to the community documentation: Kubernetes. IO/docs/setup /…

How do I adjust the nebula cluster deployment parameters?

When using Helm Install, use –set to set deployment parameters that override variables in values. Yaml in helm Chart. References: helm. Sh /docs/intro/…

Is it compatible with nebula 1.0.0+ cluster deployment?

V1.0.0 + does not support the internal DNS, you need to modify the charts/nebula/values. The yaml, configuration is as follows:

hostNetwork: true
metadEndpoints: []
  - 192.168. 02.: 9559
  - 192.168. 03.: 9559
  - 192.168. 04.: 9559
Copy the code

How can I access nebula internal components outside the K8s cluster?

NodePort is used to access the Graphd component in this article. Other access methods include hostPort, hostNetwork, Ingress, and LoadBalancer, which need to be customized according to the actual deployment environment.

How do I view the Nebula Cluster status?

Use the Kubectl get Pods –namespace= default-l app.kubernetes. IO =nebula command, or view the nebula cluster health directly on the Kubernetes Dashboard.

How do I use other types of storage?

Reference: kubernetes. IO/useful/docs/con…

Recommended reading

  • The Index practices for the distributed Graph database Nebula Graph
  • Figure Nebula Graph TTL features of the database