Windows10 Kafka Docker cluster construction

Introduction to the

Build Kafka cluster with Windows Docker Desktop

They are running

Zk is used here, just start with Docker

#First startup
docker run -dit --name zk -p 2181:2181 zookeeper

#restart
docker restart zk

#See the log
docker logs -f zk
Copy the code

Run the Kafka

Change 192.168.101.104 to your own host IP address, and check whether the logs are normal

#First startupDocker run-dit --name kafka0 -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT= 192.168.101.104:2181-e KAFKA_ADVERTISED_LISTENERS = PLAINTEXT: / / 192.168.101.104-9092 - e KAFKA_LISTENERS = PLAINTEXT: / / 0.0.0.0:9092 - t wurstmeister/kafka docker run -dit --name kafka1 -p 9093:9093 -e KAFKA_BROKER_ID=1 -e KAFKA_ZOOKEEPER_CONNECT = 192.168.101.104:2181 - e KAFKA_ADVERTISED_LISTENERS = PLAINTEXT: / / 192.168.101.104:9093 - e Note note note note note note note note note note note note note note note note note note note note note note note note note note note note note note note KAFKA_BROKER_ID KAFKA_ZOOKEEPER_CONNECT = = 2 - e 192.168.101.104:2181 - e KAFKA_ADVERTISED_LISTENERS = PLAINTEXT: / / 192.168.101.104-9094 - e KAFKA_LISTENERS = PLAINTEXT: / / 0.0.0.0:9094 - t wurstmeister/kafka
#restart
docker restart kafka0
docker restart kafka1
docker restart kafka2

#See the log
docker logs -f kafka0

#Delete the kafka
docker rm -f kafka0
docker rm -f kafka1
docker rm -f kafka2
Copy the code

test

Create copies of 3 and partitions of 5 to check whether the configuration is successful. Then launch the consumer, 0 production message on 1 and 2

#Create a copy and partitionDocker exec-ti kafka0 kafka-topics. Sh --create --zookeeper 192.168.101.104:2181 --replication-factor 3 --partitions 5 --topic TestTopic#Check the informationDocker exec-ti kafka0 kafka-topics. Sh --describe --zookeeper 192.168.101.104:2181 --topic TestTopic docker exec-ti Kafka1 kafka-topics. Sh --describe --zookeeper 192.168.101.104:2181 --topic TestTopic docker exec-ti kafka2 Kafka-topics. Sh --describe --zookeeper 192.168.101.104:2181 --topic TestTopic
#Consumption and production, the last kafka0 output can be seen in the other twoDocker exec-ti kafka1 kafka-console-consumer.sh --bootstrap-server 192.168.101.102:9093 --topic TestTopic --from -- beginning docker exec-ti kafka2 kafka-console-consumer.sh --bootstrap-server 192.168.101.104:9094 --topic TestTopic --from-beginning docker exec-ti kafka0 kafka-console-producer.sh --broker-list 192.168.101.104:9092 --topic TestTopic
#The performance testdocker exec -ti kafka0 kafka-producer-perf-test.sh --topic TestTopic --num-records 100000 --record-size 1000 --throughput 2000 --producer-props bootstrap. Servers = 192.168.101.102:9092 Docker exec-ti kafka0 Kafka-consumer-perf-test. sh --bootstrap-server 192.168.101.104:9092 --topic TestTopic --fetch-size 1048576 --messages kafka-consumer-perf-test.sh --bootstrap-server 192.168.101.104:9092 --topic TestTopic --fetch-size 1048576 --messages 100000 --threads 1Copy the code

kafka manage

After starting with Docker, visit: http://localhost:9000/, click Add Cluster, enter the first two (name and ZK address), and save

Docker run -dit -p 9000:9000 -e ZK_HOSTS="192.168.101.104:2181" hlebalbau/kafka-manager:stable http://localhost:9000/Copy the code

Refer to the link

  • 【Kafka advanced series 003】Docker environment build Kafka cluster
  • How does Kafka completely remove topics and data