new feature: hedged request support for read-only operations
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Hi team, I've suffered tail latency in my production environment, I'm wondering if you think it makes sense to support hedged request to cut down tail latency.
Reference: https://research.google/pubs/the-tail-at-scale/
### Problem and Solution
Having hedged request, which issues duplicate request (potentially to another serving host, or using another TCP connection) will improve tail latency.
Different from the timeout configurations, with hedged request there're actually multiple on-the-flight requests; the client (opendal here) serves with the earliest arriving response, and properly manages the lifecycle for all ongoing requests.
Notice hedged request should only support idempotent operations, mostly it means read-only operations, write could also be supported by they usually introduce conflict even if idempotent (i.e., database transaction abort).
Proposed solution:
Similar to timeout and retry, hedged request could also be made as a layer and guarded by features; so users are able to decide whether to opt-in based on their requirement.
### Additional Context
It's related to another feature request issue I proposed: https://github.com/apache/opendal/issues/7168
I would like to have per-operation configuration, like timeout and hedged request trigger threshold.
### Are you willing to contribute to the development of this feature?
- [ ] Yes, I am willing to contribute to the development of this feature.
Contributor guide
Research direction
Start by reviewing the existing timeout and retry layers, then read the related issue #7168 for the proposed per-operation configuration. Define how a hedged-request layer would handle read-only operations, duplicate in-flight requests, earliest responses, and cleanup of remaining requests before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, networking, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100