[Bug][broker] expired ledger cannot be cleanup when ledger consumed completely
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Describe the bug**
In some case, expired ledger data cannot be cleanup when the topic has not new message and ledger consumed completely, because the slowestReaderPosition which set to the new ledger will be cover to the old ledger by some cursor operator which like flink reset the Durable Cursor and the auto flushed dirty mark-delete position.
Also, the last expired ledger can not be clean if the topic has not active consumer and expired the message by MessageExpiryMonitor or the cursor markDeletePosition are in some old ledger when stop produce message.

**To Reproduce**
Steps to reproduce the behavior:
1. For quickly reproduce we change the config managedLedgerMaxLedgerRolloverTimeMinutes and managedLedgerMinLedgerRolloverTimeMinutes to few minutes.
2. start broker, create topic and set-message-ttl and set-retention to 4 * managedLedgerMaxLedgerRolloverTimeMinutes;
3. create Subscription and produce some message longer than set-message-ttl;
4. stop producer and wait the message expired by MessageExpiryMonitor;
5. the are two ledger. The has data ledger's all message are expired but can not be cleanup.
**Expected behavior**
All expired ledger data should be cleanup. There are many expired ledger's data can not be clean , If we have many not new message's topic. we also need migrate the ledger data when we offline one bookie.
**Additional context**
The bug can be reproduce before the pulsar version 2.10.1, the current master pr [#14672](https://github.com/apache/pulsar/pull/14672) change the behavior of rollCurrentLedgerIfFull when roll full ledger could not create new ledger, I think should resume the behavior to create new ledger.
Contributor guide
Research direction
Start by comparing the rollCurrentLedgerIfFull behavior discussed in PR #14672 with the reproduction settings for managedLedgerMaxLedgerRolloverTimeMinutes and managedLedgerMinLedgerRolloverTimeMinutes. Reproduce the MessageExpiryMonitor scenario with an idle topic, expired messages, and a fully consumed ledger. Done means expired ledger data is cleaned up, including when production stops and when a bookie is taken offline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100