The background,

In the last section, we set up a Kafka cluster. However, Kafka clusters generate all kinds of data, such as broker information, cluster information, topic information, consumer information, etc. Kafka does not provide a graphical monitoring interface. Here we use kafka-Eagle to build a kafka management monitoring interface.

Install Kafka-Eagle on MAC

1. Install JDK

slightly

2. Install Eagle

1. Download Eagle

Download link: http://download.kafka-eagle.org/

2. Decompress and configure environment variables

1. Unpack

Gz && CD kafka-eagle-bin-2.0.3&& tar -zxvf kafka-eagle-web-2.0.3-bin.tar.gzCopy the code

2. Rename

The mv kafka - eagle - web - the 2.0.3 kafka - eagleCopy the code

3. Configure environment variables

vim /etc/profile export KE_HOME=/Users/huan/soft/kafka/kafka-eagle export PATH=$PATH:$KE_HOME/bin source /etc/profile cd  ${KE_HOME}/bin chomd +x ke.shCopy the code

KE_HOME is the name of the Eagle environment variable.

Enable kafka JMX

This configuration is optional, and you can see more monitoring information when you release it. CD ${KAFKA_HOME}/bin 2. Vim kafka-server-start.sh

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then # export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G" export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"  export JMX_PORT="9099" fiCopy the code

A JMX_PORT can be assigned to an unused port.

4. Configure Eagle

CD ${KE_HOME}/conf 2. Vim system-config.properties

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# multi zookeeper & kafka cluster list
Select * from zk; select * from zk; select * from zk
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.eagle.zk.cluster.alias=cluster1
cluster1.zk.list=localhost:2181,localhost:3181,localhost:4181
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# zookeeper enable acl
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
cluster1.zk.acl.enable=false
cluster1.zk.acl.schema=digest
cluster1.zk.acl.username=test
cluster1.zk.acl.password=test123
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# broker size online list
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
cluster1.kafka.eagle.broker.size=20
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# zk client thread limit
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.zk.limit.size=25
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka eagle webui port
Kafka Eagle front-end access port
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.eagle.webui.port=9090
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka jmx acl and ssl authenticate
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
cluster1.kafka.eagle.jmx.acl=false
cluster1.kafka.eagle.jmx.user=keadmin
cluster1.kafka.eagle.jmx.password=keadmin123
cluster1.kafka.eagle.jmx.ssl=false
cluster1.kafka.eagle.jmx.truststore.location=/Users/dengjie/workspace/ssl/certificates/kafka.truststore
cluster1.kafka.eagle.jmx.truststore.password=ke123456
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka offset storage
Kafka offset is saved in Kafka. 0.10 and later kafka clusters are saved in Kafka. The previous ones are saved in ZooKeeper
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
cluster1.kafka.eagle.offset.storage=kafka
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka metrics, 15 days by default
# Enable performance monitoring. The default number of days to save data
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.eagle.metrics.charts=true
kafka.eagle.metrics.retain=15
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka sql topic records max
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.eagle.sql.topic.records.max=5000
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# delete kafka topic token
Token value to delete kafka topic
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.eagle.topic.token=keadmin
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka sasl authenticate
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
cluster1.kafka.eagle.sasl.enable=false
cluster1.kafka.eagle.sasl.protocol=SASL_PLAINTEXT
cluster1.kafka.eagle.sasl.mechanism=SCRAM-SHA-256
cluster1.kafka.eagle.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="kafka" password="kafka-eagle";
cluster1.kafka.eagle.sasl.client.id=
cluster1.kafka.eagle.blacklist.topics=
cluster1.kafka.eagle.sasl.cgroup.enable=false
cluster1.kafka.eagle.sasl.cgroup.topics=
cluster2.kafka.eagle.sasl.enable=false
cluster2.kafka.eagle.sasl.protocol=SASL_PLAINTEXT
cluster2.kafka.eagle.sasl.mechanism=PLAIN
cluster2.kafka.eagle.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="kafka" password="kafka-eagle";
cluster2.kafka.eagle.sasl.client.id=
cluster2.kafka.eagle.blacklist.topics=
cluster2.kafka.eagle.sasl.cgroup.enable=false
cluster2.kafka.eagle.sasl.cgroup.topics=
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka ssl authenticate
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
cluster3.kafka.eagle.ssl.enable=false
cluster3.kafka.eagle.ssl.protocol=SSL
cluster3.kafka.eagle.ssl.truststore.location=
cluster3.kafka.eagle.ssl.truststore.password=
cluster3.kafka.eagle.ssl.keystore.location=
cluster3.kafka.eagle.ssl.keystore.password=
cluster3.kafka.eagle.ssl.key.password=
cluster3.kafka.eagle.blacklist.topics=
cluster3.kafka.eagle.ssl.cgroup.enable=false
cluster3.kafka.eagle.ssl.cgroup.topics=
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka sqlite jdbc driver address
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#kafka.eagle.driver=org.sqlite.JDBC
#kafka.eagle.url=jdbc:sqlite:/hadoop/kafka-eagle/db/ke.db
#kafka.eagle.username=root
#kafka.eagle.password=www.kafka-eagle.org
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# kafka mysql jdbc driver address
Mysql > select * from 'eagle'
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
kafka.eagle.driver=com.mysql.jdbc.Driver
kafka.eagle.url=JDBC: mysql: / / 127.0.0.1:3306 / ke? useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
kafka.eagle.username=root
kafka.eagle.password=root
Copy the code

5. Start Eagle

ke.sh start
Copy the code

3, ke. Sh

The command	explain
ke.sh start	Start the eagle
ke.sh stop	Stop the eagle
ke.sh restart	Restart Eagle
ke.sh status	Check the Eagle status

Iv. Reference documents

Install Kafka-Eagle on MAC or Linux. Install Kafka-Eagle on Windows. 4