This is the 21st day of my participation in the August Text Challenge.More challenges in August

Introduction to the

The delivery process of the VIRTUAL machine shelf is introduced. However, the atoms of Blue Whale standard operation and maintenance do not meet our environmental requirements, so we need to develop vsphere separately.

The environment

The name of the	version	note
The blue whale	5.1.26
Standard operations	3.3.27	Github.com/Tencent/bk-…
Python Development Framework	2.0.0	Blue Whale Development framework
vcenter	5.5.0-218311.	VCenter Server 5.5 Update 2b
pyvmomi	6.7.3

Note:

1. Due to the need to develop standard operation atom separately, it is necessary to remove blue Whale’s standard operation and maintenance from the shelf, and then deploy the customized version of standard operation and maintenance developed from the source code;
2. The standard operation and maintenance need redis support, otherwise it cannot run, Tencent/ BK-SOPS
3. Atomic development of standard operations requires learning blue Whale community “early bird gets worm”

Train of thought

1. Use the vcenter custom specification manager to clone VMS from templates. During the customization process, you need to set the following parameters: VM name, VM IP address, template name, data center, cluster name, storage location, and storage name.

2. After a new VM is started, modify kernel parameters, install The Blue Whale Agent, and modify the Zabbix-Agent address, which have been configured in the template.

3. Perform asset creation and CMDB registration based on the ENTERED VM IP address.

Virtual Machine management (VSPHERE) atomic development

1. Atomic front-end development

This section describes how to define parameters for VM management and display them on the Web UI

vim vsphere_vm_create.js
(function(){
    $.atoms.vsphere_vm_create = [
        {
            tag_code: "vsphere_vm_name".type: "input",
            attrs: {
                name: gettext("Virtual machine name"),
                placeholder: gettext("New VM name"),
                hookable: true,
                validation: [
                    {
                        type: "required"
                    }
                ]
            }
        },
        {
            tag_code: "vsphere_vm_ip".type: "input",
            attrs: {
                name: gettext("Vm IP address"),
                placeholder: gettext("Vm IP address"),
                hookable: true,
                validation: [
                    {
                        type: "required"
                    }
                ]
            }
        },
        {
            tag_code: "vsphere_template_name".type: "radio",
            attrs: {
                name: gettext("Template name"),
                items: [
                    {value: "template_root", name: "root"},
                    {value: "template_app", name: "app"},
                ],
                default: "app",
                hookable: true,
                validation: [
                    {
                        type: "required"
                    }
                ]
            }
        },
        {
            tag_code: "vsphere_datacenter_name".type: "radio",
            attrs: {
                name: gettext("Data center"),
                items: [
                    {value: "unicom-idc", name: "unicom-idc"},
                ],
                default: "unicom-idc",
                hookable: true,
                validation: [
                    {
                        type: "required"
                    }
                ]
            }
        },
        {
            tag_code: "vsphere_cluster_name".type: "radio",
            attrs: {
                name: gettext("Cluster name"),
                items: [
                    {value: "unicom-ha", name: "unicom-ha"},
                    {value: "unicom-offline", name: "unicom-offline"},
                ],
                default: "unicom-offline",
                hookable: true,
                validation: [
                    {
                        type: "required"
                    }
                ]
            }
        },
        {
            tag_code: "vsphere_folder_name".type: "input",
            attrs: {
                name: gettext("Storage location"),
                placeholder: gettext("Vm Location"),
                hookable: true,
                validation: [
                    {
                        type: "required"
                    }
                ]
            }
        },
        {
            tag_code: "vsphere_datastore_name".type: "select",
            attrs: {
                name: gettext("Memory name"),
                placeholder: gettext("Memory name"),
                items: [
                    {text: "test1.datastore1", value: "uvm50.datastore1"},
                    {text: "test2.datastore1", value: "uvm51.datastore1"},
                    {text: "test3.datastore1", value: "uvm52.datastore1"},
                ],           
                hookable: true,
                validation: [
                    {
                        type: "required"}}},]})();Copy the code

The specific Web display is as follows:

2. Atomic backend development

The backend implements virtual machine creation, which consists of three parts:

1. Virtual machine clone, modified according to clone_vm.py of Pyvmomi;
2. Pyvmomi is also used to modify the IP address and host name of a custom specification.
3. After starting the VIRTUAL machine, the customized virtual machine is in the shutdown state. We need to start the operation.

The concrete implementation is as follows:

# Modify vcenter link parameters
vim vsphere.py
# -*- coding: utf-8 -*-

import logging

from pipeline.conf import settings
from pipeline.core.flow.activity import Service
from pipeline.component_framework.component import Component
# the vsphere atom needs to be extended
from pyVmomi import vim
from pyVim.connect import SmartConnect, SmartConnectNoSSL, Disconnect
import atexit
import argparse
import getpass
import json
import time

logger = logging.getLogger('celery')

__group_name__ = U "Vm Management (VSPHERE)"

#vsphere vcenter connection parameters
host = "10.10.5.88"
user = "[email protected]"
password = "xxxxxxxxxx"
port = 443
no_ssl = True
power_on = False
resource_pool = False
vsphere_datastorecluster_name = False

Basic function of vsphere
def wait_for_task(task) :
    """ wait for a vCenter task to finish """
    task_done = False
    while not task_done:
        if task.info.state == 'success':
            return task.info.result

        if task.info.state == 'error':
            print("go to vCenter")
            return task.info.result
            task_done = True     

def get_obj(content, vimtype, name) :
    """ Return an object by name, if name is None the first found object is returned """
    obj = None
    container = content.viewManager.CreateContainerView(
        content.rootFolder, vimtype, True)
    for c in container.view:
        if name:
            if c.name == name:
                obj = c
                break
        else:
            obj = c
            break

    return obj

def ip_assign(vm, vm_ip, vm_name) :
    """ Set IP address """
    adaptermap = vim.vm.customization.AdapterMapping()
    adaptermap.adapter = vim.vm.customization.IPSettings()
    adaptermap.adapter.ip = vim.vm.customization.FixedIp()
    adaptermap.adapter.ip.ipAddress = vm_ip
    adaptermap.adapter.subnetMask = "255.255.255.0"
    adaptermap.adapter.gateway = "192.168.3.1"
    #adaptermap.adapter.dnsDomain = "localhost"
    "" "DNS Settings "" "
    globalip = vim.vm.customization.GlobalIPSettings()
    globalip.dnsServerList = "114.114.114.114"
    """ Set host name """
    ident = vim.vm.customization.LinuxPrep()
    #ident.domain = "localhost"
    ident.hostName = vim.vm.customization.FixedName()
    ident.hostName.name = vm_name
    customspec = vim.vm.customization.Specification()
    customspec.nicSettingMap = [adaptermap]
    customspec.globalIPSettings = globalip
    customspec.identity = ident
    print "Reconfiguring VM Networks . . ."
    #task = get_obj([vim.VirtualMachine],vm).Customize(spec=customspec)
    task = vm.Customize(spec=customspec)
    wait_for_task(task)
    
    return True

def clone_vm(content, template, vm_name, si, datacenter_name, vm_folder, datastore_name, cluster_name, resource_pool, power_on, datastorecluster_name) :
    """ Clone a VM from a template/VM, datacenter_name, vm_folder, datastore_name cluster_name, resource_pool, and power_on are all optional. """

    # if none git the first one
    datacenter = get_obj(content, [vim.Datacenter], datacenter_name)

    if vm_folder:
        destfolder = get_obj(content, [vim.Folder], vm_folder)
    else:
        destfolder = datacenter.vmFolder

    if datastore_name:
        datastore = get_obj(content, [vim.Datastore], datastore_name)
    else:
        datastore = get_obj(
            content, [vim.Datastore], template.datastore[0].info.name)

    # if None, get the first one
    cluster = get_obj(content, [vim.ClusterComputeResource], cluster_name)

    if resource_pool:
        resource_pool = get_obj(content, [vim.ResourcePool], resource_pool)
    else:
        resource_pool = cluster.resourcePool

    vmconf = vim.vm.ConfigSpec()

    if datastorecluster_name:
        podsel = vim.storageDrs.PodSelectionSpec()
        pod = get_obj(content, [vim.StoragePod], datastorecluster_name)
        podsel.storagePod = pod

        storagespec = vim.storageDrs.StoragePlacementSpec()
        storagespec.podSelectionSpec = podsel
        storagespec.type = 'create'
        storagespec.folder = destfolder
        storagespec.resourcePool = resource_pool
        storagespec.configSpec = vmconf

        try:
            rec = content.storageResourceManager.RecommendDatastores(
                storageSpec=storagespec)
            rec_action = rec.recommendations[0].action[0]
            real_datastore_name = rec_action.destination.name
        except:
            real_datastore_name = template.datastore[0].info.name

        datastore = get_obj(content, [vim.Datastore], real_datastore_name)

    # set relospec
    relospec = vim.vm.RelocateSpec()
    relospec.datastore = datastore
    relospec.pool = resource_pool

    clonespec = vim.vm.CloneSpec()
    clonespec.location = relospec
    clonespec.powerOn = power_on

    print("cloning VM...")
    task = template.Clone(folder=destfolder, name=vm_name, spec=clonespec)

    Return the result function to determine whether the task is complete
    return wait_for_task(task)


class VsphereVMCreateService(Service) :
    __need_schedule__ = False

    def execute(self, data, parent_data) :
        vsphere_vm_name = data.get_one_of_inputs('vsphere_vm_name')
        vsphere_vm_ip = data.get_one_of_inputs('vsphere_vm_ip')
        vsphere_template_name = data.get_one_of_inputs('vsphere_template_name')
        vsphere_folder_name = data.get_one_of_inputs('vsphere_folder_name')
        vsphere_datacenter_name = data.get_one_of_inputs('vsphere_datacenter_name')
        vsphere_cluster_name = data.get_one_of_inputs('vsphere_cluster_name')
        vsphere_datastore_name = data.get_one_of_inputs('vsphere_datastore_name')

        args = {
            "host": host,
            "user": user,
            "password": password,
            "port": port,
            "no_ssl": no_ssl,
            "power_on": power_on,
            "vm_name": vsphere_vm_name,
            "template": vsphere_template_name,
            "datacenter_name": vsphere_datacenter_name,
            "cluster_name": vsphere_cluster_name,
            "vm_folder": vsphere_folder_name,
            "datastore_name": vsphere_datastore_name,
            "datastorecluster_name": vsphere_datastorecluster_name,
            "resource_pool": resource_pool
        }

        try:
            si = None
            if args['no_ssl']:
                si = SmartConnectNoSSL(
                    host=args['host'],
                    user=args['user'],
                    pwd=args['password'],
                    port=args['port'])
            else:
                si = SmartConnect(
                    host=args.host,
                    user=args.user,
                    pwd=args.password,
                    port=args.port)
            # disconnect this thing
            atexit.register(Disconnect, si)
        
            content = si.RetrieveContent()
            template = None
            
            template = get_obj(content, [vim.VirtualMachine], args['template'])

            if template:
                task_info_result = clone_vm(
                    content, template, args['vm_name'], si,
                    args['datacenter_name'], args['vm_folder'],
                    args['datastore_name'], args['cluster_name'],
                    args['resource_pool'], args['power_on'], args['datastorecluster_name'])          
                #if args.opaque_network:
                # This function is disabled
                # vm = get_obj(content, [vim.VirtualMachine], args.vm_name)
                # add_nic(si, vm, args.opaque_network)


                
                if task_info_result:
                    print task_info_result
                    
                    # Customize specifications Customize VMS
                    vm = get_obj(content, [vim.VirtualMachine], args['vm_name'])
                    task_info_result = ip_assign(vm, vsphere_vm_ip, args['vm_name'])
                    if task_info_result:
                        # Start the customized VM
                        print "PowerOn vm..."
                        vm.PowerOn()
                        # Startup completed, wait for the complete startup of the 50S
                        time.sleep(50)

                        data.set_outputs('vm_ip', vsphere_vm_ip)
                        return True
                    else:
                        print task_info_result
                        data.set_outputs('ex_data'.U "Error customizing VM")
                        return False
                else:
                    print task_info_result
                    data.set_outputs('ex_data'.U "Parameter specified for clone VM is incorrect or vm name is duplicate")
                    return False
            else:
                data.set_outputs('ex_data'.U "Vm template not found")
                return False
            
        except Exception as e:
            data.set_outputs('ex_data', e)
            logger.exception(e)          
            return False

    def outputs_format(self) :
        return [
            self.OutputItem(name=U 'VM IP address', key='vm_ip'.type='string'),
            self.OutputItem(name=U 'Error message', key='ex_data'.type='string')]class VsphereVMCreateComponent(Component) :
    name = U 'Create vm'
    code = 'vsphere_vm_create'
    bound_service = VsphereVMCreateService
    #form = settings.STATIC_URL + 'custom_atoms/vsphere/vsphere_vm_create.js'
    form = '%scityre_atoms/vsphere_vm_create.js' % settings.STATIC_URL
Copy the code

Note the above process:

1. Log printing, help us troubleshoot problems in the development process;
2. Ex_data, used to display error messages in the preceding section;
3. Finally, I set the time for starting the VM to 50 seconds.

3. End result

(1) Set parameters

(2) The vm is created

conclusion

After the above automatic creation of virtual machines, we completed the initial delivery of virtual machines, followed by adding jumpers, registering CMDB and other operations, involving the development of jumpers management (JUMP) and configuration platform customization (CMDB) atoms. This will be covered in a later article.

Note that standard plug-in development guidelines should be followed during development:

1. The group naming rule is system name (system abbreviation), for example, JOB.
2. The standard plug-in code is underlined by system name_interface name, for example, job_execute_task.
3. The background class name is hump, the rule is “standard plug-in code + inherited class name”, such as JobExecuteTaskService;
4. The front-end JS file directory should be consistent with the abbreviation of the system name, and the JS file name should be consistent with the standard plug-in code.
5. Parameter tagcode is named “system name parameter name” to ensure global uniqueness. The length should not exceed 20 characters.