Kafka Configuration 2– Configure SASl-plain authentication for Kafka under Windows

​

Kafka Configuration 1– Install and configure Kafka for Windows

Kafka Configuration 2– Configure SASl-plain authentication for Kafka under Windows

Kafka configuration 3– Configure the Kafka cluster under Windows

Kafka Configuration 4– Configure the SSL certificate for Kafka on Windows

Kafka configuration 5– Configure Kafka cluster +SASL+SSL on Windows

Kafka Configuration 6– Set and add SASL users or user permissions in Windows

Zookeeper configuration 1.1 modifying the Zookeeper Configuration File In the Zookeeper installation directory conf, open zoo. CFG and add the following configuration to the file:

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000
Copy the code

Conf In the Zookeeper installation directory conf, create the zk_server_jaas.conf file and add the following information:

Server {
     org.apache.kafka.common.security.plain.PlainLoginModule required
     username="admin"
     password="admin123456"
     user_quber="quber123456"
     user_quber1="quber123456"
     user_quber2="quber123456";
};
Copy the code

Quber, quber1, quber2, quber123456, quber is used for Kafka client to access Zookeeper. Quber is used for Kafka client to read and write quber1. Quber2: Used for writing

CMD in the Zookeeper installation directory bin, open the zkenv. CMD file and edit set ZOO_LOG4J_PROP=INFO. Add the following configuration in the next line of the CONSOLE:

set SERVER_JVMFLAGS=-Djava.security.auth.login.config=D:/Net_Program/Net_Zookeeper/conf/zk_server_jaas.conf
Copy the code

Note that the configuration is a slash, not a backslash

​

Kafka installation directory (libs) Jar lz4-java-1.6.0.jar slf4J-api-1.7.28.jar slf4j-log4j12-1.7.28.jar Snappy – Java – 1.1.7.3. Jar

Conf In the Kafka installation directory config, create the kafka_server_jaas.conf file and add the following information: kafka_server_jaas.conf

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin123456"
    user_admin="admin123456"
    user_quber="quber123456"
    user_quber1="quber123456"
    user_quber2="quber123456";
};
Copy the code

Description: In the KafkaServer section, username and password are brokers used to initiate connections to other brokers. The admin user defines all connections to the broker and all client connections authenticated by the broker, including user passwords for other brokers. The admin user must be configured otherwise an error will be reported

Conf In the Kafka installation directory config, create the kafka_client_jaas.conf file and add the following information:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="quber"
    password="quber123456";
};
Copy the code

In the KafkaClient section, username and password are used by the client to configure the client to connect to the broker. In the above configuration, the client uses the quber user to connect to the broker

In the Kafka installation directory config, open the server.properties file and add the following configuration:

Listeners = SASL_PLAINTEXT: / / 192.168.2.200:9092 advertised. Listeners = SASL_PLAINTEXT: / / 192.168.2.200:9092 # use authentication protocol Security. Intel. Broker. Protocol = SASL_PLAINTEXT SASL. # SASL mechanism enabled. Mechanisms = PLAIN Sasl. Mechanism. Intel. Broker. Protocol = PLAIN # complete authentication class authorizer. Class. The name = kafka. Security. The auth. SimpleAclAuthorizer # If no ACL (access control list) configuration is found, any action is allowed. #allow.everyone.if.no.acl.found=true super.users=User:adminCopy the code

Note: The IP addresses behind the Listeners are Intranet IP addresses advertised. The IP addresses behind the listeners are external IP addresses

In the Kafka installation directory config, go to consumer. Properties and producer.properties. Add the following configuration to the file:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
Copy the code

Id =TestGroup1 in consumer.propertie. If multiple consumer clients have different group. ids, messages cannot be consumed at the same time

Bat in the kafka installation directory bin\ Windows, open kafka-server-start.bat and add the following configuration to the SetLocal keyword in the file:

set KAFKA_OPTS=-Djava.security.auth.login.config=D:/Net_Program/Net_Kafka/config/kafka_server_jaas.conf
Copy the code

Note that the configuration is a slash, not a backslash

​

2.6, modify, kafka – the console – consumer. Bat and kafka – the console – producer. Bat in kafka bin \ Windows installation directory, Bat and kafka-console-producer.bat. Add the following configuration to the SetLocal line:

set KAFKA_OPTS=-Djava.security.auth.login.config=D:/Net_Program/Net_Kafka/config/kafka_client_jaas.conf
Copy the code

Note that the configuration is a slash, not a backslash

​

Start Zookeeper and Kafka services respectively. Run CMD as the administrator (open two Windows to start Zookeeper and Kafka services respectively). Then enter: zkServer kafka-server-start D:\Net_Program\Net_Kafka\config\server.properties

3.2 Setting User Permissions This step is to configure read, write, and group permissions for the three users (quber, quber1, and quber2) mentioned above

3.2.1. Set the read and write permissions for user quber. Run CMD as the administrator, and then enter:

kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --add --allow-principal User:quber --operation Read --operation Write --topic TestTopic1 kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --add --allow-principal User:quber --operation Read --group TestGroup1Copy the code

Note: The first command sets the “read” and “write” permissions for user quber in group TestGroup1. The second command sets the “read” and “write” permissions for user quber in group TestGroup1. If the second command does not set the “read” and “write” permissions for user quber in group TestGroup1, user quber has no access to group TestGroup1 when consuming data

3.2.2 Set read permission for quber1 run CMD as administrator, then enter:

kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --add --allow-principal User:quber1 --operation Read --topic TestTopic1 kafka-acls - the authorizer kafka. Security. Auth. SimpleAclAuthorizer -- -- authorizer - properties zookeeper. Connect = 192.168.2.200:2181 - add --allow-principal User:quber1 --operation Read --group TestGroup1Copy the code

Note: The first command gives user quber1 the “read” permission in group TestGroup1. If the second command does not give user quber1 the “read” permission in group TestGroup1, user quber1 does not have the “read” permission in group TestGroup1

3.2.3 Set the write permission for Quber2 run CMD as the administrator and enter:

kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --add --allow-principal User:quber2 --operation Write --topic TestTopic1Copy the code

Note: This command is used to set the write permission for user quber2

3.2.4 Querying user Operation rights Run CMD as the administrator and enter:

.\bin\windows\kafka-acls.bat --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --list --topic testtopic1.\ bin\ Windows \kafka-acls.bat --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --list --topic TestTopic1 or kafka-acls --authorizer Kafka. Security. Auth. SimpleAclAuthorizer -- -- authorizer - properties zookeeper. Connect = 192.168.2.200: kafka - 2181 - list of acls --authorizer-properties zookeeper.connect=192.168.2.200:2181 --list --topic TestTopic1Copy the code

Description:

The above command lists all privileged users with the subject TestTopic1

​

3.2.5 Deleting user Permissions Run CMD as administrator and enter:

.\bin\windows\kafka-acls.bat --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --remove --allow-principal User:quber --operation Read --topic TestTopic1 --force or  kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties Zookeeper. connect=192.168.2.200:2181 --remove --allow-principal User:quber --operation Read --topic TestTopic1 --forceCopy the code

Note: This command deletes the read permission of user quber from topic TestTopic1

Kafka_server_jaas.conf: kafka_server_jaas.conf: kafka_server_server_jaas.conf: Then follow steps 3.2.1 to 3.2.3 to set account permissions

Note: When consuming data on the client, if you use different accounts to consume the same topic, you need to set different accounts to different groups. If you do not start consuming data on the client, you cannot consume data on the client. After dynamically adding accounts and setting account permissions, you need to restart the Kafka service

3.3.1 Start Kafka producer (Producer) run CMD as the administrator.

. \ bin \ Windows \ kafka - the console - producer. Bat - broker - list PLAINTEXT: / / 192.168.2.200:9092 - topic TestTopic1 -- producer. Config. \ config \ producer. The properties or kafka - the console - producer - broker - list PLAINTEXT: / / 192.168.2.200:9092 --topic TestTopic1 --producer.config D:\Net_Program\Net_Kafka\config\producer.propertiesCopy the code

3.3.2 Start Kafka consumer. Run CMD as administrator.

. \ bin \ Windows \ kafka - the console - consumer. Bat - the bootstrap - server PLAINTEXT: / / 192.168.2.200:9092 - topic TestTopic1 --from -- beginning -- consumer.config. config consumer. Properties or kafka-console-consumer --bootstrap-server PLAINTEXT: / / 192.168.2.200:9092 - topic TestTopic1 - from - beginning -- consumer. Config D:\Net_Program\Net_Kafka\config\consumer.propertiesCopy the code

Note: After 3.3.1 setting, the above command can normally start the consumer

4, reference document KAFKA permission configuration PLAIN/SASL authentication: www.mscto.com/cloud/26297… Late dry | Kafka rights management of actual combat: blog.csdn.net/u013256816/…

​