apache / apache/pulsar

[Enhancement] Support getting message index in `ManagedLedger` interface

Open
#24,374 0 comments 0 reactions 1 assignee Claimed by @liangyepianzhou View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before reporting

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

1. [PIP-415](https://github.com/apache/pulsar/pull/24220) supports getting a message ID by index. This capability relies on `broker entry metadata` to obtain the message index. However, custom message layer implementations (e.g., StreamNative's Ursa engine) may not utilize broker entry metadata for this purpose. see more details in https://github.com/apache/pulsar/pull/24222#issuecomment-2921951865
2. The current implement of searching message index is inefficient, that needs to traverse all entries and check whether their indexes meet the requirements.

### Solution

1. Add a method `findMessageIndexByPosition` to the `managerLedger` interface.
2. Record the first entry index in `LedgerInfo`—the manager ledger can directly compute the index based on the message ID, significantly improving API efficiency and avoiding read amplification.

### Alternatives

Add a method

### Anything else?

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.