preface

I use SASL_SCRAM configuration as an example to explain how to quickly enable authentication and authorization mechanism in a Kafka cluster to improve the security of cluster use.

But there may be scenarios where there are multiple departments, different project groups or projects sharing the cluster, different project groups or projects using different usernames/passwords or authorizing different topic/ consumer groups, so that if we command each time, When there is a large number of user information or topic, it is extremely inconvenient to view or modify, so a visual console is needed to facilitate the operation.

kafka-console-ui

Introduction to the

Kafka-console-ui supports visual ACL management based on SASL_SCRAM mechanism.

Github address: github.com/xxd76379515…

The ACL management page is as follows:

Quick to use

packaging

git clone https://github.com/xxd763795151/kafka-console-ui.git
cd kafka-console-ui
sh package.sh
Copy the code

The deployment of

# decompression
tar -zxvf kafka-console-ui.tar.gz
# Go to the uncompressed directory
cd kafka-console-ui
Copy the code

Start, stop

# start
sh bin/start.sh
# stop
sh bin/shutdown.sh
Copy the code

Modify the configuration

# Edit configuration
vim config/application.yml
Copy the code

The main configurations are as follows:

kafka:
  config:
    # Kafka Broker addresses, separated by commas
    bootstrap-server: 'localhost:9092'
    If acl is not enabled, ignore the following items
    enable-acl: true
    SASL_PLAINTEXT = SASL_PLAINTEXT = SASL_PLAINTEXT = PLAINTEXT = PLAINTEXT
    security-protocol: SASL_PLAINTEXT
    sasl-mechanism: SCRAM-SHA-256
    The super administrator username has been configured as the super administrator on the broker
    admin-username: admin
    # Super administrator password
    admin-password: admin
    # Enable the super administrator for automatic configuration creation
    admin-create: true
    Zk address of the broker connection
    zookeeper-addr: localhost:2181
    sasl-jaas-config: org.apache.kafka.common.security.scram.ScramLoginModule required username="${kafka.config.admin-username}" password="${kafka.config.admin-password}";
Copy the code

The value of enable-acl must be true. In addition, change the broker address, ZK address, and super account password to your own.

Note that whether or not the configuration items are automatically created administrator user, if kafka cluster configuration is enabled ACL, but haven’t create the cluster node of the super administrator has started, the cluster is still unavailable state, each cluster node communication between the authentication is failed, can directly start the console, let it created automatically the super administrator, You don’t have to create the user manually.

Functional specifications

  • User Permission List

Displays information about all current user permissions and can be queried:

  • Managing Production Rights

Quickly add or delete a user’s permission to send messages to a topic

  • Managing consumption Rights

Quickly add or delete a user’s permission to subscribe to a topic using a consumer group

  • Delete the current user and all related permissions

Delete the current user configuration and all permissions granted by the user

  • Fine-grained permission control

You can select a resource (topic or consumer group) to add what permissions (white list, blacklist, etc can be configured)

  • View and manage the permission details of a resource

  • Users are added or updated

  • Viewing User Information

Note that this password is consistent, there may be inconsistent or no password, this is the effect result. Because kafka scram user passwords are unidirectionally encrypted and cannot be decrypted, the password is cached in kafka Scram. If a user is not created through kafka scram, the cached password is inconsistent or there is no cache. Therefore, every time you view user details, The cached password is checked against the actual Kafka user password to determine if the detected password is correct.

  • The administrator cannot perform this operation

Administrator information. Do not perform operations to avoid misoperations that may affect cluster stability.

At the end of the language

There are some other functions, ACL is still relatively complete, the disadvantage is only support SASL_SCRAM.

PLAINTEXT (PLAINTEXT) is not required for PLAINTEXT transmission if a Kafka cluster is deployed on an Intranet and only its own project is used. If you’re in the cloud, you’re exposed to the public network, the message security is very high and you probably need SSL channel encryption.

Consider this solution if you only do permission authentication and use the security protocol SASL_PLAINTEXT.