evo-flow creates ClickHouse Kafka Engine integration at boot even in QUEUE_MODE=direct, pointing to a non-existent broker (localhost:9092)

[ricardosantis]

Summary

evo-flow-community:1.0.0 creates the ClickHouse Kafka Engine integration (contact_events_kafka_queue, journey_trigger_kafka_queue + materialized views) at boot even when running in QUEUE_MODE=direct / WRITE_MODE=ch-sync, pointing at the hardcoded default broker localhost:9092. On deployments without Kafka/Temporal (the documented "direct" single-node mode), this leaves ClickHouse with Kafka Engine tables that continuously retry connecting to a non-existent broker: constant error spam in clickhouse-server.err.log plus a steady CPU burn.

Environment

• evo-flow-community image: evoapicloud/evo-flow-community:1.0.0
• Deployed as part of the Evo CRM Community v1.0.0 stack (Docker Swarm) with:
  • RUN_MODE=api, QUEUE_MODE=direct, WRITE_MODE=ch-sync, STORAGE_MODE=clickhouse, BROKER_TYPE=rabbitmq
  • No Kafka and no Temporal services in the stack
• ClickHouse: clickhouse/clickhouse-server:latest (26.7.3.19)
• Redis present; RabbitMQ present

What happens (logs)

Evo Flow correctly skips its own in-process Kafka consumer:

[KafkaConsumerService] Should start consumer: false, Queue is Kafka: false
[KafkaConsumerService] Kafka Consumer not started - RunMode: api, QueueMode: direct

but immediately after, the ClickHouseService still creates the Kafka integration unconditionally, using the default broker (env KAFKA_BROKERS / KAFKA_BROKERS_INTERNAL are not set):

[ClickHouseService] Creating Kafka integration for ClickHouse...
[ClickHouseService] 🔍 DEBUG - KAFKA_BROKERS_INTERNAL from env: undefined
[ClickHouseService] 🔍 DEBUG - this.config.kafka?.brokersInternal: localhost:9092
[ClickHouseService] Kafka Engine table 'evo_campaign.contact_events_kafka_queue' already points at the configured broker ('localhost:9092')
[ClickHouseService] Kafka queue table 'contact_events_kafka_queue' created
[ClickHouseService] Materialized view 'contact_events_kafka_mv' created
[ClickHouseService] Creating Journey Trigger Queue for Temporal workflows...
[ClickHouseService] Kafka table for journey triggers created

Because no Kafka broker exists on localhost:9092, ClickHouse's Kafka Engine consumers retry forever:

<Error> StorageKafka (evo_campaign.contact_events_kafka_queue): [rdk:ERROR] [thrd:localhost:9092/bootstrap]: 1/1 brokers are down
<Error> StorageKafka (evo_campaign.journey_trigger_kafka_queue): [rdk:FAIL] [thrd:localhost:9092/bootstrap]: Connect to ipv4#127.0.0.1:9092 failed: Connection refused

Root cause

In src/modules/processing/clickhouse/clickhouse.service.ts, createKafkaIntegration() (and the Journey Trigger / Temporal queue creation) run during ClickHouse initialization without checking the queue/write mode:

• createKafkaIntegration(databaseName, 'contact_events') is invoked right after createContactEventsTable(...) for every boot.
• The broker fallback in src/modules/processing/config/processing.config.ts is process.env.KAFKA_BROKERS_INTERNAL || ... || 'localhost:9092' (and KAFKA_BROKERS || 'localhost:9092'), so a stack deployed without Kafka gets tables bound to a dead broker.
• There is no env var to disable the Kafka integration creation.

Additional compatibility bug found while investigating

The Journey Trigger queue code then inspects the consumers with:

SELECT last_exception, num_messages_read FROM system.kafka_consumers
WHERE database = 'evo_campaign' AND table = 'journey_trigger_kafka_queue' AND last_exception != ''

On current ClickHouse (26.x), system.kafka_consumers has no last_exception column — exceptions live in the exceptions.text array. The query fails every boot:

Query 50c6fb16... failed: Unknown expression identifier `last_exception` in scope
SELECT last_exception, num_messages_read FROM system.kafka_consumers ...

It is caught and the service continues, but this is a latent incompatibility with modern ClickHouse releases.

Impact

• CPU: Kafka Engine consumers keep a steady ~5% CPU on a small host just retrying a connection that will never succeed (much higher during ClickHouse restarts).
• Log noise: clickhouse-server.err.log is flooded every few seconds.
• None on the direct-mode event path (contact_events via WRITE_MODE=ch-sync), because nothing ever produces to the evo-campaign-events topic and nothing consumes journey-triggers in this mode — but the phantom infrastructure is still created and retried.

Suggested fix

Gate the Kafka integration on the actual queue mode:

if (queueMode === 'kafka' || writeMode === 'kafka') {
  await this.createKafkaIntegration(databaseName, 'contact_events');
  await this.createJourneyTriggerQueue(databaseName);
}

or add an explicit opt-out env flag (e.g. KAFKA_INTEGRATION_ENABLED=false) for direct/rabbitmq deployments. Additionally, update the system.kafka_consumers introspection query to use the current schema (exceptions.text array) instead of the removed last_exception column.

Reproduction

1. Deploy the Evo CRM Community v1.0.0 Swarm stack with QUEUE_MODE=direct, WRITE_MODE=ch-sync, BROKER_TYPE=rabbitmq and no Kafka / Temporal services.
2. Boot evo-flow-community:1.0.0.
3. Observe the localhost:9092 Kafka Engine tables being created in evo_campaign and the endless broker-down errors in clickhouse-server.err.log.

Happy to test a patched image against this stack if useful.