Elasticsearch 6.3 was released on the 13th of last month with a number of new features, most notably the x-Pack integration. Elasticsearch SQL is available in the latest X-Pack, and while it’s still experimental, it’s much more user-friendly than writing complex DSL queries.

Create a Search cluster for Elasticsearch on alicun CentOS 7.4 cloud hosts. For details on how to create a search cluster for Elasticsearch on Alicun CentOS 7.4 cloud hosts, you need to create a search cluster for Elasticsearch.

Role assignment

At present, there are three cloud hosts. In order to achieve high availability, at least two Master nodes are required. Of course, since there are only three hosts, each one needs to act as a Data Node.

The host name Network IP Public IP role The HTTP service
es-master01 172.16.14.85 39.104.186.15 master & data enable
es-master02 172.16.14.86 39.104.95.206 master & data disable
es-data01 172.16.14.87 39.104.179.119 data disable

Mount data disks

In normal cases, a separate data disk is purchased to store index data when you purchase the Aliyun host to build the Elasticsearch search cluster. Therefore, you need to mount the purchased data disk first. For details of the mounting process, please refer to ali’s official tutorial. Only the commands used are posted below:

fdisk -l                                # View available data disks
fdisk /dev/vdb                          # Enter n>>p>> Enter >> Enter >>wq
fdisk -l                                Verify the data disk
mkfs.ext3 /dev/vdb1                     # format
cp /etc/fstab /etc/fstab.bak            Back up the fstab file
echo /dev/vdb1 /data ext3 defaults 0 0 >> /etc/fstab
mkdir /data                             Create a data directory
mount /dev/vdb1 /data                   # Mount data disk to data directory
df -h                                   # verify mount
Copy the code

System optimization

1. Modify the host name and Hosts

You need to change the host names of Hosts based on roles to facilitate host identification. You need to change the Hosts of Hosts to facilitate communication between nodes.

vi /etc/hostname   Change the host name

vi /etc/hosts      # modify the hosts

172.16.14.85       es-master01
172.16.14.86       es-master02
172.16.14.87       es-data01
Copy the code

2. Create an ES user

As root is not allowed or recommended to run Elasticsearch, you need to create a new user to start Elasticsearch.

adduser elastic    # create elastic user
passwd elastic     # Change elastic password
Copy the code

3. Create an ES data log folder

Example Set the data file and log directory of Elasticsearch to the data disk.

cd /data/
mkdir elastic
cd elastic
mkdir data          Create a data directory
mkdir log           Create log directory
mkdir bak           Create a backup directory
chown -R elatic /data/elastic/  # change /data/elastic/ directory owner to elastic
Copy the code

4. Optimize the number of file handles and processes available to users

The new version of Elasticsearch requires that the number of available file handles is at least 65536 and the number of processes is at least 2048.

The two error messages correspond to the following:

  1. Max file descriptors [4096] for elasticSearch process is too low, increase to at least [65536];
  2. Max number of threads [1024] for user [es] is too low, increase to at least [2048].
vim /etc/security/limits.conf * soft nofile 655350 * hard nofile 655350 * soft nproc 4096 * hard nproc 8192 elastic soft  memlock unlimited elastic hard memlock unlimitedCopy the code

5. Modify kernel switching

To avoid unnecessary disk and memory swapping that affects efficiency, change vm.swappiness to 1.

Max_map_count = max_map_count = max_map_count = max_map_count = max_map_count Max Virtual memory areas VM. Max_map_count [65530] likely too low, increase to at least [262144].

vim /etc/sysctl.conf

vm.swappiness = 1
vm.max_map_count = 655360

sysctl -p   # Effective immediately
Copy the code

6. Close the swap

swapoff -a
Copy the code

7. Restart to make the configuration take effect

reboot
Copy the code

Download decompressed files

Download the Java, Elasticsearch, and Kibana installation packages, decompress them to /opt/soft/, and change the owner of ElasticSearch-6.3.0 and Kibana-6.3.0 to Elastic.

mkdir /opt/downloads/
mkdir /opt/soft/
cd/ opt/wget downloads/wget HTTP: / / https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.0.tar.gz https://artifacts.elastic.co/downloads/kibana/kibana-6.3.0-linux-x86_64.tar.gz wget http://download.oracle.com/otn/java/jdk/8u162-b12/0da788060d494f5095bf8624735fa2f1/jdk-8u162-linux-x64.tar.gz tar -zxvf Gz -c /opt/soft/ tar -zxvf JK-8U162-linux-x64.tar. gz -c /opt/soft/ tar -zxvf Kibana-6.3.0-linux-x86_64.tar. gz -c /opt/soft/ chown -r elastic /opt/soft/ chown -r elastic / opt/soft/kibana 6.3.0 /Copy the code

Configure the Java runtime environment

su elastic                   # Switch to the elastic user
vim ~/.bash_profile          Modify only the elastic user's own environment variables

exportJAVA_HOME = / opt/soft/jdk1.8.0 _162exportJRE_HOME = / opt/soft/jdk1.8.0 _162 / jreexportThe CLASSPATH =. : / opt/soft/jdk1.8.0 _162 / lib: / opt/soft/jdk1.8.0 _162 / jre/libexport PATH=$PATH: / opt/soft/jdk1.8.0 _162 / bin: / opt/soft/jdk1.8.0 _162 / jre/binCopy the code

Configure the memory usage of Elasticsearch

Configure the JVM maximum heap and minimum heap, typically half of the server’s physical memory, up to 32GB.

cdOptions -xms2g -xmx2g /opt/soft/ elasticSearch -6.3.0/config/ vim JVMSCopy the code

Configure Elasticsearch

Modify ElasticSearch.yml using the following template

# ======================== Elasticsearch Configuration =========================
#
# ---------------------------------- Cluster -----------------------------------
#es Cluster name
cluster.name: es-cluster        

The current node name
node.name: es-mater01           

Configure the roles as master node and data node
node.master: true
node.data: true

Configure the data path and log path
path.data: /data/elastic/data
path.logs: /data/elastic/log

Enable memory lock
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
# ---------------------------------- Network -----------------------------------
# Set the communication IP address between nodes, set it as 0.0.0.0 mainly to take into account the communication between Intranet nodes, and at the same time convenient for external network developers to debugNetwork. Bind_host: 0.0.0.0# set 9200 communication IP address, generally set to external IP addressNetwork. Publish_host: 39.104.186.15Set the HTTP communication port
http.port: 9200

Set HTTP to enable
http.enabled: true

Set up to run cross-domain access
http.cors.enabled: true
http.cors.allow-origin: "*"
http.max_content_length: 500mb

# --------------------------------- Discovery ----------------------------------
[" ES-master01 ", "ES-master02 "," ES-data01 "] [" ES-master01 ", "ES-master02 "," ES-data01 "]
discovery.zen.ping.unicast.hosts: ["172.16.14.85"."172.16.14.86"."172.16.14.87"]

#discovery.zen.ping.multicast.enabled: false

discovery.zen.no_master_block: write
discovery.zen.fd.ping_timeout: 10s

Set the minimum number of primary nodes to (master_node_count+1)/2
discovery.zen.minimum_master_nodes: 2

# ---------------------------------- Gateway -----------------------------------
Set data recovery to 2 nodes
gateway.recover_after_nodes: 2
gateway.expected_nodes: 3
gateway.recover_after_time: 1m

#
# ---------------------------------- Various -----------------------------------
Disallow wildcard mode to drop index
action.destructive_requires_name: true
#indices.cluster.send_refresh_mapping: false

indices.recovery.max_bytes_per_sec: 200mb
indices.memory.index_buffer_size: 20%

#script.inline: true
#script.indexed: true

# Turn off xpack security check
xpack.security.enabled: false
Copy the code

The plugin is recommended

The first two plugins are Chrome plugins for easy access to ES, and the last one is a go client, which is also easier to use.

  1. ElasticSearch Query Builder
  2. ElasticSearch Head
  3. ElasticHD

Using Kibana

Elasticsearch can be viewed using the official Kibana action, among other plug-ins. Modify kibana. Yml in elasticsearch. Url: http://es-master01:9200 can be used, more detailed configuration can be reference.

conclusion

Elasticsearch is a search cluster that can be found on CentOS 6.4. For details about Elasticsearch, see Elasticsearch 6.3.

Any Code, Code Any!

Scan code to pay attention to “AnyCode”, programming road, together forward.