confluentinc / confluentinc/librdkafka
The connection from producer to Kafka cluster is flapping
- Dominant language
- C
- Stars
- 1k
- Forks
- 3.3k
- Avg merge
- 5d 12h
- Merged PRs (30d)
- 6
Description
The Kafka is deployed in a kubernetes environment with 3 pods in a kafka cluster.
The producer kafka connection keeps flapping. The error callback is producing below low -
**err: Local: Broker transport failure
reason: kafka-0.kafka-headless.common.svc.cluster.local:9092/1001: Receive failed: Disconnected**
What is the reason for this flap?
I have below config set, but i am not getting any additional information.
**rd_kafka_conf_set(conf, "debug", "broker,topic,msg", NULL, 0);**
**rd_kafka_conf_set(conf, "log_level", "7", NULL, 0);**
Other config:
rd_kafka_conf_set(conf, "socket.keepalive.enable", "true", NULL, 0);
rd_kafka_conf_set(conf, "log.connection.close", "true",NULL, 0);
Contributor guide
Research direction
Start with the reported rd_kafka_conf_set debug, log_level, socket.keepalive.enable, and log.connection.close settings, then inspect the broker and Kubernetes logs for kafka-0.kafka-headless.common.svc.cluster.local:9092. Done means identifying a reproducible cause for the disconnects and documenting the log evidence or configuration change that prevents the connection from flapping.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, kubernetes
- Domain
- distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100