KRAFT mode does not work in cluster

[sparrowV]

I've got cluster of 2 nodes:

---
version: '2'
services:

  broker:
    image: confluentinc/cp-kafka:7.0.0
    hostname: broker
    container_name: broker
    ports: 
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
      KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092'
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: localhost
      KAFKA_PROCESS_ROLES: 'broker,controller'
      KAFKA_NODE_ID: 1
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
      KAFKA_LISTENERS: 'PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
    volumes:
      - ./update_run.sh:/tmp/update_run.sh
    command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'"

  broker2:
        image: confluentinc/cp-kafka:7.0.0
        hostname: broker2
        depends_on:
          - broker
        container_name: broker2
        ports:
          - "9093:9093"
          - "9104:9104"
        environment:
          KAFKA_BROKER_ID: 2 
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
          KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker2:29092,PLAINTEXT_HOST://localhost:9093'
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
          KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
          KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
          KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
          KAFKA_JMX_PORT: 9104
          KAFKA_JMX_HOSTNAME: localhost
          KAFKA_PROCESS_ROLES: 'broker'
          KAFKA_NODE_ID: 2
          KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
          KAFKA_LISTENERS: 'PLAINTEXT://broker2:29092,CONTROLLER://broker2:29093,PLAINTEXT_HOST://0.0.0.0:9093'
          KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
          KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
          KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
        volumes:
          - ./update_run.sh:/tmp/update_run.sh
        command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'"

docker-compose up gives following error:

any thoughts why INCONSISTENT_CLUSTER_ID in FETCH response is happening?
@aesteve @ybyzek any thoughts?

[aesteve]

I think I faced this before yes.

In /tmp/update_run.sh you might have some random cluster id or something like that?
Like the random-uuid here

When you're using two brokers, you can't use a random cluster_id for each of the brokers (that leads to the error you're seeing: "inconsistent cluster id"), in that case, you'd need to use a "fixed" cluster id of your choice for both brokers.

Be careful though, that the cluster id must be properly formatted. For example here I used one I picked from the docs iirc.

If you want to use your own valid cluster_id, you can try:

docker run --rm -it --entrypoint /bin/kafka-storage confluentinc/cp-kafka:7.2.1 random-uuid

and change update-run.sh to use it (as I linked before)

Works on my machine™

Alternatively, you could refer to an environment variable in update_run.sh (like CLUSTER_ID or KAFKA_CLUSTER_ID or something) and set it in your docker-compose.yml along with other env variables there. Whatever you prefer.

[techjourney010]

Checkout the following article. It can help you to run a Kafka Cluster in KRaft mode.

[techjourney010]

I think the issue should be closed.