[Bug] broker: Stale backlog after creating Earliest subscription on trimmed topic
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [x] I understand that [unsupported versions](https://pulsar.apache.org/contribute/release-policy/#supported-versions) don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### User environment
pulsar broker 3.0.12
java 17
### Issue Description
Observed on a 3.0.x based branch.
### What did you expect to see?
After the inactive subscription is deleted and all old ledgers are trimmed, recreating the durable subscription with `Earliest` should initialize the cursor with backlog `0` if there is no real remaining data.
`stats.backlog`, `stats-internal`, and precise backlog should be consistent.
### What did you see instead?
`admin topics stats` can report `backlog > 0`, while:
- `admin topics stats-internal` shows old ledgers already trimmed
- precise backlog is `0`
- the consumer cannot actually receive any message
- unloading the topic makes the reported backlog become `0`
### Error messages
```text
```
### Reproducing the issue
1. Create a partitioned topic.
2. Produce messages across multiple ledgers.
3. Configure retention aggressively so old ledgers are trimmed, leaving only a new empty current ledger.
5. Create a new and durable subscription name and `SubscriptionInitialPosition.Earliest`.
6. Compare:
- `admin topics stats `
- `admin topics stats-internal `
- precise backlog
### Additional information
Root cause is in `ManagedLedgerImpl#getFirstPositionAndCounter()` during earliest cursor initialization.
When all historical ledgers are already trimmed and only an empty current ledger remains, `getFirstPosition()` can return a synthetic position on the old last confirmed ledger (for example `469:-1`).
`getFirstPositionAndCounter()` then uses that synthetic position in `getNumberOfEntries(Range.openClosed(...))`, which counts entries from a ledger that no longer exists in `ledgers`.
This leads to an incorrect `messagesConsumedCounter` when a durable subscription is recreated with `SubscriptionInitialPosition.Earliest` after inactive subscription deletion. The inaccurate counter then propagates into approximate backlog stats.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
Research direction
Start at ManagedLedgerImpl#getFirstPositionAndCounter() and reproduce the scenario with a trimmed topic, a new durable Earliest subscription, and the admin topics stats commands. Compare stats, stats-internal, and precise backlog; done when an empty current ledger produces consistent zero backlog and the consumer receives no phantom messages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100