[Bug] Persisted batch ACK indexes are not restored from MetadataStore
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [x] I understand that [unsupported versions](https://pulsar.apache.org/contribute/release-policy/#supported-versions) don't get bug fixes. I reproduced the issue on the current `master` branch.
### User environment
- Broker version: current `master` (`67205b946b763d43ba254601514f2f42ee5c520e`)
- Broker operating system: macOS / Darwin 25.6.0, x86_64
- Broker Java version: Eclipse Temurin 25.0.2
- Client library: Java client built from the same `master` revision
- Reproduced with both a focused Managed Ledger test and a broker-level `Shared` subscription test
### Issue Description
When a cursor is cleanly closed through the MetadataStore persistence path, its partial batch acknowledgment indexes are written to `ManagedCursorInfo.batchedEntryDeletionIndexInfos`. However, reopening the cursor from that metadata does not restore those indexes.
For example, given a batch containing `A`, `B`, `C`, and `D`, if `A` and `B` are acknowledged while `C` and `D` remain unacknowledged, unloading and reloading the topic can cause `A` and `B` to be delivered again.
Expected behavior:
- The reopened cursor restores the persisted batch ACK bitmap.
- Only `C` and `D` are delivered after recovery.
Actual behavior:
- The persisted batch ACK bitmap is absent from the reopened cursor.
- Already acknowledged messages such as `A` and `B` can be redelivered.
This is a bug because the write path persists the state, and the equivalent BookKeeper cursor-ledger recovery path restores it correctly. The MetadataStore recovery path restores the mark-delete position, individually deleted entry ranges, and properties, but skips `batchedEntryDeletionIndexInfos` before completing cursor initialization.
### Error messages
```text
No exception or log error is emitted. The persisted batch ACK state is silently omitted during cursor recovery.
```
### Reproducing the issue
1. Enable batch-index-level acknowledgment persistence.
2. Create a `Shared` subscription and publish one batch containing four messages: `A`, `B`, `C`, and `D`.
3. Consume the batch and acknowledge only `A` and `B`; wait for the acknowledgments to complete.
4. Cleanly unload the topic so that the cursor position is persisted through MetadataStore (`cursorsLedgerId == -1`).
5. Verify before reopening that `ManagedCursorInfo.batchedEntryDeletionIndexInfos` contains the partial batch ACK record.
6. Create a new client and reopen the subscription, ensuring that a new cursor instance is constructed.
7. Inspect the recovered cursor or consume the remaining messages.
The recovered cursor returns `null` for the batch position ACK set even though the metadata contained it before reopening. At the broker level, already acknowledged messages can consequently be delivered again.
The issue does not require `dispatcherPauseOnAckStatePersistentEnabled` and does not require exceeding the batch ACK persistence limit. A normal consumer reconnect can reuse the in-memory cursor and therefore may not exercise the affected recovery path.
### Additional information
The asymmetry is in `ManagedCursorImpl`:
- `persistPositionMetaStore(...)` includes partial batch ACK records through `buildBatchEntryDeletionIndexInfoList()`.
- The BookKeeper cursor-ledger recovery branch calls `recoverBatchDeletedIndexes(...)`.
- The `cursorsLedgerId == -1` MetadataStore recovery branch does not call it.
A fix and regression tests are available in [PR #26474](https://github.com/apache/pulsar/pull/26474).
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
Research direction
Start in ManagedCursorImpl, comparing persistPositionMetaStore(...) with the BookKeeper recovery branch and the cursorsLedgerId == -1 MetadataStore path. Run the focused Managed Ledger test and broker-level Shared subscription test described in the issue; done means recovery preserves the partial batch ACK state and does not redeliver acknowledged messages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 20/100