preface

Kafka supports two authentication mechanisms based on SSL and SASL. SASL is used in this article.

Kafka supports five SASL mechanisms: GSSAPI, PLAIN, SCRAM, OAUTHBEARER, and Delegation Token.

This article uses SCRAM authentication: a user name/password authentication mechanism, can be dynamically added and deleted (PLAIN is also user name/password authentication, but write dead in the configuration file, add new configuration needs to restart)

Kafka version I’m using the latest code pulled from the Github Trunk branch.

Starting build with version 3.1.0-SNAPSHOT using Gradle 7.1.1, Java 1.8 and Scala 2.13.6

Be aware of the version, some configurations may need to be adjusted, and direct replication may not be correct.

certification

  1. Adding an administrator
sh bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config 'SCRAM-SHA-256=[password=admin],SCRAM-SHA-512=[password=admin]' --entity-type users --entity-name admin
Copy the code

New administrator: admin/admin. Zk is started at this point, but the broker is not started because the user information is stored on ZK.

In the new version, most commands are recommended to use –bootstrap-server instead of –zookeeper. This command can be used to start the broker before it is started. You can also use –bootstrap-server to add administrator users. Then stop the broker.

  1. Enabling ACL Configuration

Add the following configuration to server.properties:

listeners=SASL_PLAINTEXT://localhost:9092
sasl.enabled.mechanisms=SCRAM-SHA-256
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
security.inter.broker.protocol=SASL_PLAINTEXT
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
super.users=User:admin; User:xuxiaodong
Copy the code

Super. users: Configure the super administrator. For example, two users are configured: admin and Xuxiaodong. The previous new administrator command only adds the admin User.

  1. Configure the jaas

Create the kafka-broker.jaas file in the config directory as follows:

KafkaServer {
org.apache.kafka.common.security.scram.ScramLoginModule required
username="admin"
password="admin";
};
Copy the code

If there are multiple brokers, steps 2 and 3 require this configuration for each broker.

  1. Start the broker

Change the startup script bin/kafka-server-start.sh and add the following startup parameters somewhere in the first few lines:

export KAFKA_OPTS=" $KAFKA_OPTS -Djava.security.auth.login.config=/Users/xuxd/SourceCode/github/kafka/kafka/config/kafka-broker.jaas
Copy the code

The JAAS file for step 3 needs to be specified in the configuration.

The configuration on the server side is complete. Start the broker.

  1. New users

Add two users: writer (for sending messages to topic) and Reader (for consuming messages).

sh bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config 'SCRAM-SHA-256=[password=writer],SCRAM-SHA-512=[password=writer]' --entity-type users --entity-name writer --command-config cmd-config
 
 
sh bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config 'SCRAM-SHA-256=[password=reader],SCRAM-SHA-512=[password=reader]' --entity-type users --entity-name reader --command-config cmd-config 
Copy the code

Notice that there is a –command-config cmd-config at the end of the command. This is because after permission authentication is enabled, you need to configure authentication information when running the command. Otherwise, the command will not be processed normally.

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin";
Copy the code

authorization

The new user and authorization are separate and need to be configured separately. The admin user is already specified as the super administrator in the broker configuration file.

However, writer and Reader users are not authorized to send or consume messages, and they are not designated as super administrators. You need to configure permissions for them separately.

  1. Assign the writer user the permission to send messages to the test_topic topic
sh bin/kafka-acls.sh --bootstrap-server 'localhost:9092' --add --allow-principal User:"writer" --producer --topic 'test_topic' --command-config cmd-config
Copy the code
  1. Give the reader user permission to consume test_TOPic_consumer messages with the test_TOPic_consumer consumer group
sh bin/kafka-acls.sh --bootstrap-server 'localhost:9092' --add --allow-principal User:"reader" --consumer --topic 'test_topic' --group 'test_topic_consumer' --command-config cmd-config
Copy the code

After the configuration is complete, the following permission information is displayed:

Client uses authentication

Instead of using the script to send or consume the message, you can use the client code to verify the message. The code example is not provided, and you can find it in any search:

Message sending If no permission is configured, message sending fails: The client logs are as follows:

Logs on the server are as follows:

To configure SASL authentication information for producer, add the following configurations:

        props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_PLAINTEXT");
        props.put(SaslConfigs.SASL_MECHANISM, "SCRAM-SHA-256");
        props.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"writer\" password=\"writer\";");
Copy the code

Send the message again and it will be sent normally. On the consumer end, add the preceding parameters to the user name and password, and grant the same user the permission to send and consume resources.