[Bug] [ml] Incorrect behavior of Topic Retention Policy
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
master branch
### Minimal reproduce step
```java
public void testRetentionInMB() throws Exception {
@Cleanup("shutdown")
ManagedLedgerFactory factory = new ManagedLedgerFactoryImpl(metadataStore, bkc);
ManagedLedgerConfig config = new ManagedLedgerConfig();
config.setRetentionSizeInMB(2);
config.setRetentionTime(-1, TimeUnit.SECONDS);
config.setMaxEntriesPerLedger(2);
config.setMinimumRolloverTime(0, TimeUnit.SECONDS);
@Cleanup
ManagedLedgerImpl ml = (ManagedLedgerImpl) factory.open("retention_test_ledger", config);
@Cleanup
ManagedCursor c1 = ml.openCursor("c1");
// 512K per entry, 2 entries per ledger, 1 ledger has 1MB data.
byte[] content = new byte[1024 * 512];
for (int i = 0; i < 21; i++) {
ml.addEntry(content);
}
// should be 11 ledgers.
assertEquals(ml.ledgers.size(), (21 / 2) + 1);
long firstKey = ml.ledgers.firstKey();
// ack 5 ledgers.
c1.markDelete(new PositionImpl(firstKey + 5, 0));
// trigger trim ledgers manually
CompletableFuture f = new CompletableFuture<>();
ml.trimConsumedLedgersInBackground(false, f);
f.get();
}
```
1. For the test, we set the retention policy by size(2MB); 2 entries per ledger and each entry 512KB, each ledger is 1MB.
2. Add 21 entries to the ManagedLedger, it will open 11 ledgers(I assume these ledger ID are **[1, 11]**).
3. Set markDeletePosition to **6:0**, which means **[1, 5]** ledgers are consumed
4. Trigger trim ledger task manually
5. Wait until trim task finish
### What did you expect to see?
Ledgers **[1, 3]** deleted.
### What did you see instead?
Ledgers **[1, 5]** deleted
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Assessment
This issue has not been assessed yet.