redpanda-data / redpanda-data/console

[BUG] Consumer group overview fails when `ListOffsets` returns `LEADER_NOT_AVAILABLE`

Open
#2,573 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
4.3k
Forks
432
Avg merge
3d 6h
Merged PRs (30d)
40

Description

The consumer group overview fails when Redpanda Console calculates consumer lag and ListOffsets returns a partial LEADER_NOT_AVAILABLE error.
This seems similar to issue #2327, where LISTENER_NOT_FOUND causes the same code path to fail. In both cases, one partition cannot return an end offset while the remaining partitions are still healthy.

Environment
  • Redpanda Console: current master branch
  • Deployment mode: Kubernetes
  • Kafka cluster: 3 brokers
  • One broker unavailable, example:
kafka-cluster-1-0-vxh5k  0/1  Init:ImagePullBackOff
kafka-cluster-1-1-xr9zv  1/1  Running
kafka-cluster-1-2-g8zrs  1/1  Running
What happened
Image

Console logs:

failed to list end offsets from some shards request ListOffsets has 1 separate shard errors, first: LEADER_NOT_AVAILABLE: There is no leader for this topic-partition as we are in the middle of a leadership election.

In my setup, one broker is unavailable. Topics with a replication factor of 1 that were hosted on that broker no longer have a leader because a new leader cannot be elected, example:

Topic: test_dash Partition: 0 Leader: none Replicas: 0 Isr: 0

Console calculates lag by fetching committed group offsets and then calling ListEndOffsets to fetch partition high watermarks. If one partition cannot return an end offset, Console cannot calculate lag for that partition, but it can still show consumer group data for healthy partitions. The old behavior treats the partial ListEndOffsets/ListOffsets failure as fatal, which can hide the whole consumer group overview.

Expected behavior

Image

Actual behavior

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the consumer group overview's lag calculation, specifically the committed-offset and ListEndOffsets/ListOffsets path described in the issue. Compare the handling with issue #2327 and reproduce with one unavailable broker. Done means healthy partitions still appear when one partition returns LEADER_NOT_AVAILABLE, while the affected partition's lag is handled visibly rather than hiding the whole overview.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, typescript
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.