hoangsonww / hoangsonww/FastFetch-API-Fetch-Enhancer

Hedged Requests + Alt-Endpoint Failover + Priority Hints

Open
#3 0 comments 0 reactions 1 assignee Claimed by @hoangsonww View on GitHub
documentation enhancement good first issue help wanted question
Dominant language
TypeScript
Stars
12
Forks
8
PR merge metrics
No merged PRs in 30d

Description

# Feature: **Hedged Requests** + **Alt-Endpoint Failover** + **Priority Hints**

**Summary**
Reduce tail latency and improve reliability by (1) issuing **hedged requests** (a second request after a small delay if the first stalls), (2) **failing over** to configured alternate endpoints/regions when primary hosts are degraded, and (3) supporting **priority hints** to inform the scheduler and hedging policy.

## Motivation

* Long-tail outliers (cold starts, noisy neighbors) hurt UX even when averages look fine.
* Many APIs expose multiple regions/hosts; automatic failover avoids manual retries.
* Not all requests are equal—**priority** should shape timeouts, hedging delays, and retry budgets.

## Goals

1. **Hedging**: launch a backup request after `hedgeDelayMs` if the first hasn’t received headers; cancel the loser on winner resolve.
2. **Alt-Endpoint Failover**: per-host list of alternates (e.g., `api-us`, `api-eu`, `api-ap`); choose next on network error, 5xx, or breaker-open; preserve method/body/headers.
3. **Priority Hints**: `priority: "high" | "normal" | "low"` adjusts default `timeoutMs`, `hedgeDelayMs`, and retry/backoff caps.

## API (additive)

```ts
import { fastFetch, configureFastFetch } from "@hoangsonw/fast-fetch";

configureFastFetch({
hedging: {
enabled: true,
hedgeDelayMs: 200, // when to launch hedge if no headers yet
maxHedges: 1, // number of backup attempts per call
cancelLoser: true // abort the slower in-flight
},
failover: {
enabled: true,
policy: "sequential", // "sequential" | "round-robin" | "random"
map: {
"https://api.example.com": [
"https://api-eu.example.com",
"https://api-ap.example.com"
]
},
failOnStatuses: [500, 502, 503, 504, 522, 524], // overridable
respectCircuitBreaker: true
},
priorities: {
high: { timeoutMs: 8000, hedgeDelayMs: 120, retryBudget: { maxAttempts: 5 } },
normal: { timeoutMs: 15000, hedgeDelayMs: 200, retryBudget: { maxAttempts: 4 } },
low: { timeoutMs: 25000, hedgeDelayMs: 350, retryBudget: { maxAttempts: 2 } }
}
});
```

Per-request overrides:

```ts
await fastFetch("/orders?id=123", {
priority: "high",
hedging: { hedgeDelayMs: 100 }, // override
failover: { policy: "round-robin" } // override
});
```

## Behavior Details

* **Hedging lifecycle**:

* Start request A → if no headers by `hedgeDelayMs`, start request B (same endpoint).
* First to receive headers “wins”; losing request is aborted (if `cancelLoser`).
* Hedging counts toward retry budget; integrates with existing backoff/timeout logic.
* **Failover**:

* On qualifying failure (network, breaker-open, or `failOnStatuses`), swap base URL with next alternate and replay request.
* Preserves method/body; re-signing hooks can be supported via user-provided `beforeSend` callback (future).
* **Priority**:

* Sets sensible defaults for timeout, hedging delay, and retry budget.
* Users can still override per-request.

## Examples

**Kill tail latency on product page**

```ts
const res = await fastFetch("/api/product/42", {
priority: "high",
hedging: { hedgeDelayMs: 120 }
});
```

**Global failover map**

```ts
configureFastFetch({
failover: {
enabled: true,
map: {
"https://api.mycorp.com": [
"https://api-eu.mycorp.com",
"https://api-ap.mycorp.com"
]
}
}
});
```

## Acceptance Criteria

* [ ] Hedged requests launch after delay and cancel the loser; only one `Response` is resolved to the caller.
* [ ] Failover switches to alternates on qualifying errors and respects circuit-breaker state.
* [ ] Priorities adjust defaults as configured; per-request overrides take precedence.
* [ ] Works with existing **dedup**, **timeouts**, **backoff**, and (optional) **circuit breaker** features.
* [ ] Unit tests: hedging win/lose paths, failover sequencing, priority defaulting, and interaction with AbortSignals.
* [ ] README updated with guidance and caveats (idempotency, signing, metrics).

## Notes & Caveats

* **Idempotency**: Hedging duplicates requests—safe for GET/HEAD; for mutating verbs require idempotency keys or disable hedging.
* **Auth/Signing**: If headers or bodies are time-bound (HMAC, nonce), expose a `beforeSend(req)` hook (follow-up) to re-sign on failover/hedge.
* **Metrics**: Consider optional event hooks (`onHedge`, `onFailover`, `onWinner`) for observability (separate issue).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.