dell / dell/iDRAC-Telemetry-Reference-Tools

Kafka Pump does not auto-reconnect when partition leader fails/changes

Open
#166 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
44
Forks
32
PR merge metrics
No merged PRs in 30d

Description

**Description:**

While testing iDRAC Telemetry in a multi-broker Kafka HA setup, we observed that the Kafka Pump does not automatically reconnect to a new leader when the current partition leader fails/changes even though the pump is configured with a headless service.

Image

**Steps to Reproduce:**

- Configure Kafka with multiple brokers and replication factor = 3.
- Start the Kafka Pump and have it write to a specific partition (e.g., partition 0).
- Kill the leader broker for that partition.
- Observe that the Kafka Pump stops sending data.

**Expected Behavior:**
The Kafka Pump should detect the partition leader change and automatically reconnect to the new leader broker, resuming data ingestion without manual intervention.

**Actual Behavior:**
The Kafka Pump still tries to push data to the dead broker remains disconnected from the topic/partition until it is restarted or manually redirected.

**Impact:**

Breaks continuous telemetry ingestion in HA scenarios.
Requires manual intervention, reducing system reliability.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.