apache / apache/pulsar

msgbacklog is not accurate between shutdowns

Open
#15,948 6 comments 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
the msgBacklog is not accurate after restarts and request msgBacklog for subscription even specify --get-precise-backlog

**To Reproduce**
Steps to reproduce the behavior:
1. get msgBacklog by 'pulsar-admin topics partitioned-stats --get-precise-backlog --get-subscription-backlog-size "
alias ts="pulsar-admin topics stats --get-precise-backlog --get-subscription-backlog-size'
2. restart broker
3. request for backlog again, or consume msg in a particular subscirption
4. we can consume more message than the 'msgBacklog'

**Expected behavior**
the msgBacklog can indicate the precise backlog in a subscription in any case.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the discrepancy with the `pulsar-admin topics partitioned-stats` and `topics stats` commands using `--get-precise-backlog` and `--get-subscription-backlog-size`, then restart a broker and compare the results while consuming messages. Trace the precise subscription-backlog handling across broker restarts. Done means `msgBacklog` remains accurate after shutdowns and matches the messages that can still be consumed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.