Before starting deployment, it is recommended to switch to the domestic image source according to Ubuntu, which can save a lot of precious time!

Kafka’s distributed scheduling is depend on the Zookeeper to complete, so there’s no deployment Zookeeper friend, you can refer to my another blog morning ops notes | deploy Ubuntu Zookeeper cluster, first deployment in Zookeeper cluster oh!

The sample machine

The host	IP
kafka-01	192.168.0.1
kafka-02	192.168.0.2
kafka-03	192.168.0.3

Install the Kafka

In this example, kafka-01 is configured to be the same as kafka-01, and the rest of the machine is configured to be the same as kafka-01.

$ sudo mkdir /data/kafka && cd ~
$ wget "https://downloads.apache.org/kafka/2.7.1/kafka_2.12-2.7.1.tgz"$ curl http://kafka.apache.org/KEYS | gpg --import $ wget https://dist.apache.org/repos/dist/release/kafka/2.7.1/kafka_2.12-2.7.1.tgz.asc $GPG -- verify kafka_2. 12-2.7.1. TGZ. Asc Kafka_2.12-2.7.1.tgz $sudo tar-xvzf kafka_2.12-2.7.1.tgz --directory /data/kafka --strip-components 1 $sudo rm -rf Kafka_2. 12-2.7.1. TGZ kafka_2. 12-2.7.1. TGZ. Asc# # https://tecadmin.net/install-apache-kafka-ubuntu/ for more reference
Copy the code

Modifying a Configuration File

$ sudo mkdir /data/kafka-logs
$ sudo cp /data/kafka/config/server.properties{,.bak}
$ sudo vim /data/kafka/config/server.properties
 
broker.id=0    # Each host is differentListeners = PLAINTEXT: / / 0.0.0.0:9092 advertised. Listeners = PLAINTEXT: / / IP < > : 9092 delete. Topic. Enable =true
leader.imbalance.check.interval.seconds=5  # Leader imbalance check intervalleader.imbalance.per.broker.percentage=1 log.dirs=/data/kafka-logs offsets.topic.replication.factor=3 The retention. Hours = 72 log. Segment. Bytes = 1073741824 zookeeper. Connect = 192.168.0.1:2181192168 0.2:2181192168 0.3:2181# zk cluster address
  
$ sudo vim /data/kafka/bin/kafka-server-start.sh
export JMX_PORT=12345    Expose JMX port for monitoring
Copy the code

Register as a Systemd service

$ sudo adduser --system --no-create-home --disabled-password --disabled-login kafka
$ sudo chown -R kafka:nogroup /data/kafka
$ sudo chown -R kafka:nogroup /data/kafka-logs
  
$ sudo vim /etc/systemd/system/kafka.service
[Unit]
Description=High-available, distributed message broker
After=network.target
[Service]
User=kafka
ExecStart=/data/kafka/bin/kafka-server-start.sh /data/kafka/config/server.properties
[Install]
WantedBy=multi-user.target

## Enable the service
$ sudo systemctl enable kafka.service
$ sudo systemctl start kafka.service

# # https://kafka.apache.org/quickstart for more reference
Copy the code

Test Kafka

$ /data/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
$ /data/kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181
  
$ /data/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
> Hello World
  
# Another terminal
$ /data/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
Hello World
Copy the code

Deploy CMAK

CMAK (formerly known as Kafka Manager) is a tool for managing Apache Kafka clusters.

CMAK can be downloaded here: github.com/yahoo/CMAK

$ cd/ data & sudo wget https://github.com/yahoo/CMAK/releases/download/3.0.0.5/cmak-3.0.0.5.zip $sudo unzip cmak - 3.0.0.5. Zip $sudo mv cmak - 3.0.0.5 / data/cmak $sudo chown -r kafka: nogroup/data/cmak $sudo vim/data/cmak/conf/application. Conf kafka-manager.zkhosts="192.168.0.1:2181192168 0.2:2181192168 0.3:2181" # zk cluster
basicAuthentication.enabled=true
basicAuthentication.username="<username>"
basicAuthentication.password="<password>"

# Register as Systemd service for later operation and maintenance
$ sudo vim /etc/systemd/system/kafka-manager.service
[Unit]
Description=High-available, distributed message broker manager
After=network.target
[Service]
User=kafka
ExecStart=/data/kafka-manager/bin/kafka-manager
[Install]
WantedBy=multi-user.target

## Enable the service
$ sudo systemctl enable kafka-manager.service
$ sudo systemctl start kafka-manager.service
Copy the code

Then visit http://192.168.0.1:9000 to open the ‘Kafka’ management interface.

If this blog has been helpful to you, please remember to leave a comment + like + bookmark.

I am Chen, on the road of technology we forge ahead together!