Azure / Azure/azure-sdk-for-java
[BUG]readMany suffers from head-of-line blocking due to Flux.mergeSequential in ParallelDocumentQueryExecutionContext
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 2.2k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 178
Description
**Describe the bug**
`CosmosAsyncContainer.readMany` uses `ParallelDocumentQueryExecutionContext` for partition ranges with 2+ items. This execution context merges per-partition query results using `Flux.mergeSequential(obs, fluxConcurrency, prefetch)`.
`Flux.mergeSequential` preserves upstream ordering by only requesting new items from the source iterator when the head (earliest-in-order) inner publisher completes (FluxMergeSequential.drain() → s.request(1)). If a non-head inner publisher completes, no new subscription
is triggered — the freed concurrency slot sits idle.
**For `readMany`, this causes severe head-of-line blocking: a single slow partition response stalls all remaining partition fetches, even when concurrency slots are available.**
**My use-case based on cosmos db diagnostics log**
`readMany` with 207 items spanning 55 partition key ranges, fluxSequentialMergeConcurrency = 6.
One partition (pkRangeId=92) had a transitTime of 982ms (backend latency was only 0.69ms — the delay was network/IO, not Cosmos server-side). The other 5 concurrent partition queries completed in 1-13ms each.
**Expected: the 5 freed slots immediately dispatch the remaining 27 partition queries.** Total wall clock ≈ 982ms + 27×3ms/5 ≈ ~1000ms.
**Actual: no new partition queries were dispatched for 981ms until pkRangeId=92 completed.** Only then did the remaining 27 queries start. Total wall clock = 1,020ms, of which 981ms was 5 idle concurrency slots.
Timeline (55 cross-partition queries, 6 point reads omitted):
Time (ms) In-flight Event
───────────────────────────────────────────
0.0 6 Initial 6 partition queries dispatched
0.5 – 17.0 6 Queries complete, head advances, slots refilled — working correctly
17.0 6 pk=92 starts as part of a new batch of 6 (pk=91,92,79,80,69,70)
17.5 – 18.5 6→5 pk=91 (head before 92) completes, pk=95 dispatched to refill slot
19.0 – 31.0 5→4→3→2→1 pk=70,80,79,95,69 complete — slots NOT refilled (pk=92 is now head)
31.0 – 999.0 1 Only pk=92 in-flight. 5 slots idle. 27 queries waiting.
999.0 6 pk=92 completes. Head advances through 5 already-done inners.
s.request() called 6 times. 6 new queries dispatched.
999.5 – 1020 6→... Remaining 27 queries drain normally.
**Root cause**
`readMany` does not require ordered results — it returns items as a collection with no ordering guarantee. However, it reuses `ParallelDocumentQueryExecutionContext`, which uses `Flux.mergeSequential` to support deterministic ordering for general cross-partition queries (important for continuation tokens / pagination).
`readMany` pays the head-of-line blocking cost of `mergeSequential` for ordering semantics it does not need.
Contributor guide
Assessment
This issue has not been assessed yet.