In today’s article, I’ll walk you through how to automate the deployment of The Elastic Stack using Ansible. For developers who are not familiar with Ansible, this tutorial provides a primer on Ansible. This is a series of articles:

  • How to deploy Elastic Stack-Overview using Ansible
  • Deploying Elastic Stack-ElasticSearch with Ansible
  • Deploying Elastic Stack-Kibana with Ansible
  • Deploying Elastic Stack-Security with Ansible
  • Using Ansible to automate Deployment of Elastic Stack-metricBeat

At the end of the day, you can find all the deployment files: github.com/liu-xiao-gu…

 

What is a Ansible?

Ansible is a new automated o&M tool developed based on Python. It integrates the advantages of many o&M tools (Puppet, Chef, func, and Fabric) to implement batch system configuration, batch program deployment, and batch command running. Ansible was developed based on Paramiko and works on a modular basis, with no bulk deployment capability of its own. The real bulk deployment is the module that Ansible runs on, ansible just provides a framework. Ansible does not require client/agents to be installed on remote hosts because they communicate with remote hosts over SSH. Ansible, which has been officially acquired by Red Hat, is one of the most recognized automation tools available and is easy to use and learn. Is one of the skills that every operation and maintenance engineer must master.

 

In today’s experiment, I will use a native Mac OS and a native Ubuntu OS 20.04 to demonstrate. In real life, you can use servers in the cloud to do this, although the deployment is slightly different.

I will install Ansible on Mac OS. I will use Ansible to remotely deploy Webserver, Elasticsearch and Kibana on Ubuntu OS machines.

 

Install Ansible

Installing Ansible on Mac OS is very simple. We must install Python first, and then we can use the following command to do so:

pip3 install ansible
Copy the code

After installation, we can use the following command to view ansible help information:

ansible --help
Copy the code

 

Establishing an SSH Connection

Remote installation is implemented using SSH. In order for Ansible to work properly, you must copy your local SSH certificate to a remote machine, which eliminates the need for passwords to communicate. For some cloud platforms, we can copy the local Ansible machine’s certificate to the relevant configuration. If you don’t have your own SSH configuration, follow these steps to generate a certificate on your machine. In my case, I first produce SSH certificates on my MacOS machine using the following command:

ssh-keygen
Copy the code

The command above will generate the following certificate in the.ssh directory on the machine:

$ pwd
/Users/liuxg/.ssh
$ ls
id_rsa.pub   id_rsa     
Copy the code

Above, we can see the newly generated ID_rsh. pub and id_RSA certificates. Where id_rsh.pub is the public key certificate. We use the following command to copy the certificate to an Ubuntu OS machine:

ssh-copy-id remote_user@remote_IP
Copy the code

In my case, I run the following command on my MacOS machine:

ssh-copy-id liuxg@ubuntu
Copy the code

Above, liuxg is my account on Ubuntu OS.

$ ssh-copy-id liuxg@ubuntu
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
liuxg@ubuntu's password: 

Number of key(s) added:        1

Now try logging into the machine, with:   "ssh 'liuxg@ubuntu'"
and check to make sure that only the key(s) you wanted were added.
Copy the code

Using the command above, we can view the following authorized_keys file in the.ssh directory of Ubuntu OS:

$ cd .ssh
$ pwd
/home/liuxg/.ssh
liuxg@liuxgu:~/.ssh$ ls
authorized_keys  id_rsa.pub  id_rsa 
Copy the code

You can open this file. You can see that it is exactly the same as in id_rsa.pub on Mac OS. This indicates that the certificate on Mac OS has been successfully copied.

Next, we typed the following command on the Mac OS machine:

ssh liuxg@ubuntu
Copy the code

From the above we can see that we can log in to the Ubuntu OS machine without entering any password. This shows that our configuration is successful.

 

Test drive – Deploy a Webserver

In this section, we will deploy an Apache Webserver from MacOS to an Ubuntu machine using Ansible.

For developers who are not familiar with Ansible, let’s start by deploying a simple Webserver to familiarize themselves with Ansible deployment. First we create a directory as follows:

cd
mkdir -p ansible/elasticsearch
cd ansible/elasticsearch
Copy the code

Above, we create a directory called ansible/ ElasticSearch in the user’s home directory. This directory is the working directory for our ElasticSearch project. In this directory, we first create a subdirectory called inventory and create a file called hosts.yml in this directory. The contents of the hosts.yml file are as follows:

inventory/hosts.yml

[elk]
192.168.0.4

[webservers]
192.168.0.4

[all]
192.168.0.4
Copy the code

Above, you can change the IP address according to your own configuration. As shown in the figure above, I will deploy ELK to the server described by ELK, and I will deploy Webserver to the server represented by WebServers. If we want to deploy a configuration to all machines, we can select All. The service here is a list and we can add more servers to the list above.

We also created two additional subdirectories under elasticSearch: Playbooks and Roles:

$ pwd
/Users/liuxg/ansible/elasticsearch
$ ls
inventory playbooks roles
Copy the code

Next, go to the roles directory and type the following command:

$ pwd
Users/liuxg/ansible/elasticsearch/roles
$ ansible-galaxy init webserver
Copy the code

This creates a subdirectory called WebServer under the roles directory:

In the WebServer directory, it creates a template directory and files. Let’s open the files subdirectory and create a file like this:

index.html

<html>
    <body>
        <h1>This is a sample web page</h1>
    </body>
</html>
Copy the code

The index. HTML file above is what we will need to display in the future when the Apache server is started. Next, we open the Tasks file directory:

We open the main.yml file and type the following:

main.yml

---
# tasks file for webserver

- name: install apache2
  apt: 
    name: apache2
    state: latest
    update_cache: yes

- name: deploy index.html
  copy:
    src: index.html
    dest: /var/www/html/index.html

- name: starting apache2
  service:
    name: apache2
    state: started
    enabled: yes
Copy the code

As shown above, in the main.yml file, we created three tasks. It shows the contents of the task in the name above. The name is descriptive for our own reference. Under each task, there are commands to execute. These commands constitute the functionality that needs to be accomplished. Up there, we did three things:

  1. Install the latest apache2
  2. Copy the local index.html file to the appropriate directory on the target machine
  3. Start the Apache service to make the Web Server work properly

Next, we create a file called deploy-webServer. yml in our playbooks directory:

deploy-webserver.yml

--- # This playbook will deploy webserver - hosts: webservers become: yes roles: - .. /roles/webserverCopy the code

As shown above, we will deploy the WebServer into the server described by the WebServers. These WebServers are described in the inventory/hosts.yml section described earlier. The Playbook will use roles/ WebServer roles.

Next, we use the following command to deploy the webServer:

ansible-playbook -K -i inventory/hosts.yml playbooks/deploy-webserver.yml
Copy the code

Note that we used the -k parameter above. This is because on the remote machine Ubuntu OS, we need to provide the password of the sudo command to complete the installation. If you don’t want to enter a password, you can refer to the article “How to set and Use sudo Password for Ansible Vault”.

The command output is as follows:

The above shows that all command runs were successful.

We can check if our Apache server has been successfully installed by typing the following command in the browser of the MacOS computer http://ubuntu:80:

It shows that our Apache server is completely successful. With this simple tutorial, you learned how to use Ansible for remote deployment.

So far, we have successfully deployed a WebServer. Let’s review an entire step:

  1. Install ansible
  2. Create the hosts.yml file
  3. Create roles and update Tasks
  4. Create the playbook
  5. The deployment of

In the rest of the tutorial, you will install Elasticsearch separately by following the steps above. Read the article “How to Automate Elastic Stack Deployment using Ansible (Part 2)”.

Reference:

【 1 】 elastic/ansible – elasticsearch