Nebula Graph’s Ansible practice

NebulaGraphCommunity first published by NebulaGraph NebulaGraphCommunity, nebulagraph takes a look at the technical practice of nebulagraph database

background

During Nebula-Graph’s daily testing, we often deployed Nebula-Graph on the server. In order to improve efficiency, we need a tool that can help us achieve rapid deployment. The main requirements are:

• By deploying Nebula Graph with a non-root account, we can set cGroup resource limits for this user.
• Configuration files can be changed on the operator and then distributed to the deployed cluster for easy testing of various callbacks.
• It can be called using a script, so that we can inherit it from the platform or tool we test later.

In terms of tool selection, the early tools are Fabric and Puppet, while the newer tools are Ansible and SaltStack.

Ansible has a 40K+ Star on GitHub and was acquired by Red Hat in 2015 with an active community. Many open source projects provide deployment methods for Ansible, such as KubesPray in Kubernetes and Tidb-Ansible in TiDB.

Taken together, we deployed Nebula Graph using Ansible.

Introduce Ansible

The characteristics of

Ansible is an open source, automated deployment tool (Ansible Tower is commercial). It has the following characteristics:

• The default protocol is based on SSH and requires no additional agent deployment compared to SaltStack.
• Use Playbook, Role, and Module to define the deployment process.
• Operation behavior is idempotent.
• Modular development, module rich.

Advantages and disadvantages are obvious.

• The advantage of using SSH is that most machines can deploy using Ansible by default as long as they have an account password, but the disadvantage is that the performance is poor.
• Using Playbook to define the deployment process and Python’s Jinja2 as the template rendering engine is convenient for those who are familiar with it, but adds to the learning cost for those who are not.

To sum up, it is suitable for batch deployment of small batch machines and does not need to worry about additional agent deployment, which matches our requirements.

Deployment of logic

Typically, machines can be divided into three roles for offline deployment.

• Ansible machine: A machine running Ansible must be able to connect to all machines using SSH.
• An extranet resource machine: Runs tasks that need to be connected to the Internet, such as downloading RPM packages.
• Server: The server that runs the service, which can be network isolated and deployed by executing machine

The task logic

There are three main levels of tasks in Ansible:

• Module
• Role
• Playbook

Modules, including CoreModule and CustomerModule, are basic units of Ansible tasks.

Ansible generates a new Python file based on the module’s code, which is stored in the TMP folder using SSH. The output will be returned using SSH, and the remote directory will be deleted.

#The TMP file is not deleted
export ANSIBLE_KEEP_REMOTE_FILES=1

#- VVV Displays debugging informationAnsible -m ping all-VVv <192.168.8.147> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'the User = "nebula" -o ConnectTimeout ControlPath = = 10 - o/home/vesoft /. Ansible/cp/d94660cf0d - tt 192.168.8.147'/bin/sh -c '"' "'/usr/bin/python/home/nebula /. Ansible/TMP/ansible - 045877 / TMP - 1618982672.9659252-5332-61577192 AnsiballZ_ping. Py && sleep 0'"'"''Copy the code

As you can see, ansiballz_ping. py is the Python file generated from the Module. Log in to that machine and execute Python statements to see the result.

python3 AnsiballZ_ping.py

#{"ping": "pong", "invocation": {"module_args": {"data": "pong"}}}
Copy the code

Standard output for running a Python file is returned, and Ansible does additional processing on the returned result.

A Role is a series of tasks in a series of modules that can be registered to pass context parameters.

Typical example:

1. Create a directory
2. If the directory is created successfully, continue the installation. Otherwise, exit the deployment project.

Playbook is an association between an organization’s deployment machine and roles.

By grouping different machines at Inventory and deploying different roles for different groups, you can achieve very flexible installation and deployment tasks.

Once playBook is defined, different environments can perform the same deployment process by simply changing the machine configuration in Inventory.

Module custom

A custom filter

Ansible uses Jinja2 as the template rendering engine. You can use Jinja2’s built-in filter, for example

#If the default filter is used, the default output is 5

ansible -m debug -a 'msg={{ hello | default(5) }}' all
Copy the code

Sometimes we need custom filters to manipulate variables, and a typical scenario is the address of Nebula – Metad -meta_server_addrs.

• When there is only 1 Metad, the format ismetad1:9559.
• When there are three metad, the format ismetad1:9559,metad2:9559,metad3:9559

Create a map_fomat.py Python file using the filter_plugins directory.

# -*- encoding: utf-8 -*-
from jinja2.utils import soft_unicode

def map_format(value, pattern) :
    """ e.g. "{{ groups['metad']|map('map_format', '%s:9559')|join(',') }}" """
    return soft_unicode(pattern) % (value)

class FilterModule(object) :
    """ jinja2 filters """
    def filters(self) :
        return {
            'map_format': map_format,
        }
Copy the code

{{groups [‘ metad] | map (‘ map_format ‘, ‘% s: 9559’) | join (‘, ‘)}} is the value we want.

The custom module

Custom modules need to conform to Ansible framework format, including parameters, standard return, error return, etc.

If you have written a custom module, configure ANSIBLE_LIBRARY in ansible. CFG for Ansible to obtain it.

Specific reference website: ansible – docs. Readthedocs. IO/useful/stable – 2…

Nebula Graph’s Ansible practice

Because Nebula Graph itself is not complex to launch, deploying Nebula-Graph using Ansible is straightforward.

1. Download the RPM package.
2. Copy the RPM package to the deployment machine, decompress it, and save it to the destination folder.
3. Update the configuration file.
4. Boot through the shell.

Use generic roles

Nebula Graph has three components: Graphd, Metad, and Storaged. The three components are named and launched in the same format, using common roles. Graphd, Metad, and Storaged reference common roles, respectively.

On the one hand, it is easier to maintain, and on the other, the services deployed are more fine-grained. For example, if storaged is deployed on machine A, B and C, and graphd is deployed on machine C, then machine A and B will not have the graphd configuration file.

#Generic role, using the variable install/task/main.yml
- name: config {{ module }}.conf
  template:
    src: "{{ playbook_dir}}/templates/{{ module }}.conf.j2"
    dest: "{{ deploy_dir }}/etc/{{ module }}.conf"
    
#Graphd role, pass the variable in nebula- Graphd /task/main.yml
- name: install graphd
  include_role:
    name: install
  vars:
    module: nebula-graphd
Copy the code

In playbook, graphd’s machine group runs graphd’s role. If A B is not in graphd’s machine group, the graphd configuration file will not be uploaded.

After this deployment, you cannot use Nebula-Graph’s Nebula. Service Start All for full startup because the configuration file for Nebula- Graphd.conf will not be available on some machines. Similarly, in playBook, you can use parameters to specify different machine groups and pass different parameters.

# playbook start.yml
- hosts: metad
  roles:
    - op
  vars:
    - module: metad
    - op: start

- hosts: storaged
  roles:
    - op
  vars:
    - module: storaged
    - op: start

- hosts: graphd
  roles:
    - op
  vars:
    - module: graphd
    - op: start
Copy the code

This is equivalent to SSH for several times to execute the startup script. Although the execution efficiency is not as good as start all, the service can be started and stopped more flexibly.

End playbook with vars_prompt

When you want to update the binary and not delete the data directory,

You can add vars_prompt to remove playbook. If the prompt is confirmed, data will be deleted. Otherwise, the playbook will exit.

# playbook remove.yml
- hosts: all
  vars_prompt:
    - name: confirmed
      prompt: "Are you sure you want to remove the Nebula-Graph? Will delete binary only  (yes/no)"
 
  roles:
    - remove
Copy the code

In ROLE, the value of the second validation is checked

# remove/task/main.yml--- - name: Information debug: msg: "Must input 'yes', abort the playbook " when: - confirmed ! = 'yes' - meta: end_play when: - confirmed ! = 'yesCopy the code

If the value is not yes, the PlayBook execution will be cancelled, removing only the binary and not the nebula cluster data.

Ac graph database technology? To join the Nebula Exchange Group, fill out your Nebulae card and Nebula Assistant will bring you into the group

Recommended reading

• [Application and challenge of ten-billion-level graph data in Kuaishou Security Intelligence]
• [Meituantu database platform construction and business practice]
• Nebula Graph’s Data Governance Practices at WeBank
• Selection of Graph database | Migration history of graph database in 360 Department of Data