cloudflare / cloudflare/cloudflare-docs
Semantics question: in AI Gateway analytics, is a retried provider call one request or two — and do cache hits count?
- Dominant language
- MDX
- Stars
- 5.2k
- Forks
- 16.7k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 337
Description
**Background.** AI Gateway logs every proxied request with status, token usage, cost, and duration, and can retry timed-out provider calls (request-handling docs), fall back across providers (Universal Endpoint), and serve responses from cache.
**Question or gap.** Scenario: 1 client prompt → provider 429/timeout → gateway retry succeeds → 2 provider-side log entries, 1 completion delivered. Later, the identical prompt is served from cache with no provider call. Across those three events, what does "requests" mean in Analytics — client requests, provider attempts, or log rows? For Costs and billing, are cached hits and failed attempts included in request counts or in token/cost totals? A team reporting "we served N requests for $X" gets materially different numbers depending on the answer (here: 3, 2, or 1 requests), and the docs don't seem to define the unit per metric.
**Proposed direction.** A short docs paragraph defining the counting unit per analytics metric (client request vs. provider attempt vs. cache hit), or an explicit dimension separating attempts from client requests, would make the numbers auditable.
**Non-goal.** Not pitching anything; we hit this while defining internal usage metering and want Cloudflare's intended semantics.
Contributor guide
Research direction
Start with the request-handling docs and Universal Endpoint documentation, then locate the AI Gateway Analytics and Costs or billing metric definitions. Compare how client requests, provider attempts, failed calls, and cache hits are represented. Done means documenting the counting unit for each affected metric or identifying an explicit attempts-versus-client-requests dimension.
Written by the indexing model from the issue text.
Assessment
- Domain
- analytics, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100