redpanda-data / redpanda-data/console

Lag and partitions offsets are presented incorrectly

Open
#788 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend frontend ui/ux
Dominant language
TypeScript
Stars
4.3k
Forks
432
Avg merge
3d 6h
Merged PRs (30d)
40

Description

It looks like console reports total lag and partitions incorrectly.

I have one consumer running in one thread consuming one topic with 10 partitions:
image

As you can see, console understands that all 10 partitions are assigned, but in the list below it reports assigned partitions: 8.
The list also only contain 8 partitions and not 10.

I believe that in the list it only shows partitions for which offsets are committed, completely ignoring partitions that haven't been consumed yet.

This screenshot of the Members tab supports this hypothesis:
image

It shows all 10 partitions, but reporting lag: 0 for the partitions that my consumer hasn't consumed from or committed offsets for.

Therefore Total lag metric is unreliable and somehow semi-random.

Expected behavior:

  • Both lists having the correct number of partitions
  • Lag is displayed as:
    • top watermark - consumer offset for partitions that have offsets committed (does it currently)
    • top watermark - bottom watermark or top watermark - 0 for the partitions that the consumer hasn't committed offsets for
    • Total lag is calculated as a sum of the above.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the consumer-group Members tab and the partition list, reproducing the case with one consumer, one topic, and 10 partitions where only 8 have committed offsets. Trace how assigned partitions, committed offsets, and watermarks are combined. Done means both lists show all assigned partitions, uncommitted partitions use the specified lag calculation, and Total lag sums those values.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.