Today I set up a ClickHouse cluster to share with you

The clickHouse cluster relies on ZooKeeper, so we need to build a ZooKeeper cluster. Due to limited resources, ZooKeeper uses a single point mode. If you have enough resources, you can build a cluster. I also recommend building a ZooKeeper cluster

  • Avoid close login
  • They are deployed
  • Clickhouse installation
  • Clickhouse configures the cluster
  • The results show

1. Exempt login

To facilitate file transfer, the two machines have been configured with encrypted login.

  • Configure the public key of node A on node B
  • Node A requests A login request from node B
  • Node B uses the public key of node A to encrypt A random text
  • Node A decrypts it using its private key and sends it back to node B
  • B verifies that the text is correct

Run the ssh-keygen -t rsa command for each server

The two machines then exchange public keys:

Copy the /root/.ssh/public key for node1 to /root/.ssh/authorized_keys for node2

Copy the /root/.ssh/public key for node2 to /root/.ssh/authorized_keys for node1

2. They are deployed

Download the ZooKeeper service and unzip it

Tar -zxvf zookeeper-3.4.9.tar.gz. C.. /servers/Copy the code

CFG file. If you want to configure the ZooKeeper cluster, you can Google it

3. Clickhouse installation

Both servers are centos. For Ubuntu or other services, check clickhouse

sudo yum install yum-utils sudo rpm --import https://repo.clickhouse.tech/CLICKHOUSE-KEY.GPG sudo yum-config-manager - add - repo https://repo.clickhouse.tech/rpm/stable/x86_64 if you want to use the latest version, please replace stable with testing (we recommend that you only used in the test environment). Prestable is also sometimes available. Then run the command: sudo yum install clickhouse-server clickhouse-clientCopy the code

Run the systemctl start clickhouse-server command to enable the clickhouse-server command on node1 and node2 respectively. Run the systemctl status clickhouse-server command to view the clickhouse-server status The clickhouse-client command can be used to check whether the installation was successful:Both machines need to be checked

4. Clickhouse cluster configuration

Modify /clickhouse-server/config. XML file on node1.

Amend section 1:

<! Listen_host ::</listen_host> <! -- Same for hosts with disabled ipv6: --> <! - < listen_host > 0.0.0.0 < / listen_host > -- > <! -- Add external configuration file metrika.xml -->Copy the code

Amendment 2:

<remote_servers> <cluster_2hards_1replicas> <! - shard 1 - > < shard > < up > < host > 172.17.162.84 < / host > < port > 9000 < / port > < up > < / shard > <! -- Shard 2 --> <shard> <replica> <host>172.17.162.83/host> <port>9000</port> </replica> </shard> </luster_2hards_1replicas> ..... Omit original configuration </emote_servers>Copy the code

Amendment 3:

< zooKeeper > <node> <host>172.17.162.83</host> <port>2181</port> </node> </ zooKeeper >Copy the code

Synchronize config.xml to /etc/clickhouse-server/ on node2

Restart the service with the systemctl restart clickhouse-server command (both machines need to be restarted)

The cluster uses port 9000 to communicate with each other. Therefore, run the Telnet command to check whether the port is enabled

5. Results presentation

Create local tables on both nodes

CREATE TABLE default.cluster2s1r_local(`id` Int32,`website` String,`wechat` String,`FlightDate` Date,Year UInt16)ENGINE = MergeTree(FlightDate, (Year, FlightDate), 8192);
Copy the code

Create distributed tables on Node1

CREATE TABLE default.cluster2s1r_all AS cluster3s1r_local ENGINE = Distributed(cluster_2shards_1replicas, default, cluster3s1r_local, rand());
Copy the code

The data is then inserted into the distributed table

INSERT INTO default.cluster2s1r_all (id, website, wechat, FlightDate, Year) values (1, 'https://niocoder.com/', 'Java dry', '2020-11-28', 2020); INSERT INTO default.cluster2s1r_all (id,website,wechat,FlightDate,Year)values(2,'http://www.merryyou.cn/','javaganhuo','2020-11-28',2020); INSERT INTO default.cluster2s1r_all (id,website,wechat,FlightDate,Year)values(3,'http://www.xxxxx.cn/','xxxxx','2020-11-28',2020);Copy the code

Then query distributed tables and local tables separately

Select * from cluster2s1r_all; SELECT * FROM cluster2s1r_all Query id: 19 bb300d e6a - bd51 b586-4-10 bf12e85eef ┌ ─ ─ id ┬ ─ website ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ wechat ─ ─ ─ ┬ ─ FlightDate ─ ┬ during ─ ─ ┐ │ │ 1 https://niocoder.com/ │ Java dry │ │ │ 2020 2020-11-28 └ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ┘ ┌ ─ ─ id ┬ ─ website ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ wechat ─ ─ ─ ─ ─ ┬ ─ FlightDate ─ ┬ during ─ ─ ┐ │ │ │ 2 http://www.merryyou.cn/ javaganhuo │ The 2020-11-28 2020 │ │ └ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ┘ ┌ ─ ─ id ┬ ─ website ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ wechat ─ ┬ ─ FlightDate ─ ┬ during ─ ─ ┐ │ │ │ 3 http://www.xxxxx.cn/ XXXXX │ │ │ 2020 2020-11-28 └ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ┘ # :) the select * from the surface of node1 cluster2s1r_local select * from cluster3s1r_local Query id: 88604512-cce5-457e-9d7f-f5ebb8d44ad6 Ok. 0 rows in set. Elapsed: Select * from cluster2s1r_local select * from cluster3s1r_local Query ID: 9 a93 ca8-3338-499 - e - 44832-07448 e7ae698 ┌ ─ ─ id ┬ ─ website ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ wechat ─ ─ ─ ┬ ─ FlightDate ─ ┬ during ─ ─ ┐ │ │ 1 https://niocoder.com/ │ Java dry │ │ │ 2020 2020-11-28 └ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ┘ ┌ ─ ─ id ┬ ─ website ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ wechat ─ ─ ─ ─ ─ ┬ ─ FlightDate ─ ┬ during ─ ─ ┐ │ │ │ 2 http://www.merryyou.cn/ javaganhuo │ The 2020-11-28 2020 │ │ └ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ┘ ┌ ─ ─ id ┬ ─ website ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┬ ─ wechat ─ ┬ ─ FlightDate ─ ┬ during ─ ─ ┐ │ │ │ 3 http://www.xxxxx.cn/ XXXXX │ │ │ 2020 2020-11-28 └ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ┘Copy the code

I hope it can help other students who have the same problem with me. This way is definitely no problem. If you have any problem, please leave a message to me