[client] Preserve lookup query ordering across retries
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Description
Lookup completion order is not strictly preserved today. `LookupSender` allows multiple in-flight requests, and a failed query is re-enqueued only after its asynchronous failure callback runs. A later query may therefore already be sent or completed before the earlier query is retried.
Historical lookup throttling adds a retry backoff. A throttled query remains in the retry queue until its next retry time, while `LookupQueue` may continue draining and sending later queries from the regular queue. This does not create the first possible out-of-order completion, but it widens the existing window and adds an explicit path for later queries to overtake a delayed retry.
For example, given submission order `A` then `B`:
1. `A` receives `HistoricalLookupThrottledException` and is delayed by retry backoff.
2. `B` is drained from the regular queue and completes.
3. `A` is retried after its backoff expires.
The completion order is then `B` followed by `A`.
We should determine whether the lookup API requires an ordering guarantee. If it does, define the required scope, such as per key, table bucket, or table, and preserve that ordering across retries without blocking unrelated lookups. Otherwise, document that lookup completion order is not guaranteed.
### Willingness to contribute
- [ ] I am willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing LookupSender and LookupQueue, focusing on asynchronous failures, retry backoff, and how regular and retry queues are drained. Establish the lookup API's required ordering scope; done means either preserving that order across retries without blocking unrelated lookups, with coverage for the A-before-B scenario, or documenting that completion order is not guaranteed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100