How can I set kafka ACLs for brokers?-CodePudding

I have some problems with configuring Kafka Acls brokers.

I'm using bitnami docker-compose-cluster.yml for my project and I want set authentication for each broker

I created kafka_jass.conf file with this content:

kafkabroker {
        security.protocol=SASL_PLAINTEXT
        sasl.mechanism=PLAIN
        sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required
        username="alice"
        password="******";
};

and added these lines to docker compose for each broker:

KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT,SASL_PLAINTEXT:PLAINTEXT
KAFKA_CFG_LISTENERS=CLIENT://:29092,EXTERNAL://0.0.0.0:9092,SASL_PLAINTEXT://broker1:9095
KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://broker1:29092,EXTERNAL://******:9092,SASL_PLAINTEXT://localhost:9095
security.inter.broker.protocol=SASL_PLAINTEXT

and this line to server.properties:

authorizer.class.name=kafka.security.authorizer.AclAuthorizer

after starting the docker compose, I receive this error for each broker:

[2022-12-06 06:47:19,679] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
broker1    | org.apache.kafka.common.KafkaException: Exception while loading Zookeeper JAAS login context [java.security.auth.login.config=/opt/bitnami/kafka/config/kafka_jaas.conf, zookeeper.sasl.client=default:true, zookeeper.sasl.clientconfig=default:Client]
broker1    |    at org.apache.kafka.common.security.JaasUtils.isZkSaslEnabled(JaasUtils.java:67)
broker1    |    at kafka.server.KafkaServer$.zkClientConfigFromKafkaConfig(KafkaServer.scala:79)
broker1    |    at kafka.server.KafkaServer.<init>(KafkaServer.scala:149)
broker1    |    at kafka.Kafka$.buildServer(Kafka.scala:73)
broker1    |    at kafka.Kafka$.main(Kafka.scala:87)
broker1    |    at kafka.Kafka.main(Kafka.scala)
broker1    | Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
broker1    |    Line 2: expected [controlFlag]

Update question:

this is docker compose content:

version: "2"

services:
  zookeeper:
    image: dockerhub.charisma.tech/bitnami/zookeeper:3.8
    hostname: zookeeper,SASL_PLAINTEXT://localhost:9091
    container_name: zookeeper
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - KAFKA_OPTS="-Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Djava.security.auth.login.config=/opt/bitnami/kafka/config/zookeeper-server.jaas"
    volumes:
      - ./config:/opt/bitnami/kafka/config

  kafka-0:
    image: dockerhub.charisma.tech/bitnami/kafka:3.2
    hostname: broker1
    container_name: broker1
    ports:
      - '9092:9092'
    volumes:
      - ./config/broker1:/bitnami
      - ./config/broker1/kafka/config/server.properties:/bitnami/kafka/config/server.properties
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://0.0.0.0:29092,EXTERNAL://0.0.0.0:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://broker1:29092,EXTERNAL://*****:9092
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_INTER_BROKER_LISTENER_NAME=SASL_PLAINTEXT
      - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
      - KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_LISTENER_NAME_EXTERNAL_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_LISTENER_NAME_EXTERNAL_PLAIN_SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required user_admin='admin-secret' user_producer='producer-secret' user_consumer='consu>      
      - KAFKA_CFG_LISTENER_NAME_CLIENT_SASL_ENABLED_MECHANISMS=PLAIN
      - KAFKA_CFG_LISTENER_NAME_CLIENT_PLAIN_SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required user_broker='broker-secret' username='broker' password='*****';"
      - KAFKA_CFG_SUPER_USERS="User:broker;User:admin"
      - KAFKA_CFG_ALLOW_EVERYONE_IF_NO_ACL_FOUND="false"
      - KAFKA_CFG_ZOOKEEPER_SET_ACL="true"
      - KAFKA_CFG_OPTS="-Djava.security.auth.login.config=/opt/bitnami/kafka/config/kafka-server.jaas"
    depends_on:
      - zookeeper

CodePudding user response：

Your JAAS file is improperly formatted. The first two options are Kafka broker properties, not JAAS entries.

This is the minimum information you need, but you can add additional users here, too.

Refer. https://docs.confluent.io/platform/current/kafka/authentication_sasl/index.html

KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
        username="alice"
        password="******";
};

Regarding your container, you cannot edit the server properties directly; that's what the environment variables already do. Use KAFKA_CFG_AUTHORIZER_CLASS. Similarly, the inter broker protocol needs uppercased and prefixed appropriately for the bitnami container to use it.

Also, look at the README of the bitnami Kafka container and it already has ways to configure authentication and user accounts via extra environment variables.