cloudflare / cloudflare/cloudflare-docs

Semantics question: in AI Gateway analytics, is a retried provider call one request or two — and do cache hits count?

Open
#32,983 9 comments 0 reactions 0 assignees View on GitHub
product:ai-gateway
Dominant language
MDX
Stars
5.2k
Forks
16.7k
Avg merge
2d 6h
Merged PRs (30d)
337

Description

**Background.** AI Gateway logs every proxied request with status, token usage, cost, and duration, and can retry timed-out provider calls (request-handling docs), fall back across providers (Universal Endpoint), and serve responses from cache.

**Question or gap.** Scenario: 1 client prompt → provider 429/timeout → gateway retry succeeds → 2 provider-side log entries, 1 completion delivered. Later, the identical prompt is served from cache with no provider call. Across those three events, what does "requests" mean in Analytics — client requests, provider attempts, or log rows? For Costs and billing, are cached hits and failed attempts included in request counts or in token/cost totals? A team reporting "we served N requests for $X" gets materially different numbers depending on the answer (here: 3, 2, or 1 requests), and the docs don't seem to define the unit per metric.

**Proposed direction.** A short docs paragraph defining the counting unit per analytics metric (client request vs. provider attempt vs. cache hit), or an explicit dimension separating attempts from client requests, would make the numbers auditable.

**Non-goal.** Not pitching anything; we hit this while defining internal usage metering and want Cloudflare's intended semantics.

Contributor guide

Open the contributing guide

Research direction

Start with the request-handling docs and Universal Endpoint documentation, then locate the AI Gateway Analytics and Costs or billing metric definitions. Compare how client requests, provider attempts, failed calls, and cache hits are represented. Done means documenting the counting unit for each affected metric or identifying an explicit attempts-versus-client-requests dimension.

Written by the indexing model from the issue text.

Assessment

Domain
analytics, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.