0xPlaygrounds / 0xPlaygrounds/rig

feat: capture provider-reported cost from OpenAI-compatible streaming responses

Aperta
#2,242 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Rust
Stelle
8.6k
Fork
959
Merge medio
4h 32m
PR unite (30g)
117

Descrizione

## Problem

Several OpenAI-compatible gateways report the monetary cost of a request in the chat-completions stream, but rig drops it. Notably **OpenCode Go** emits a dedicated `inference-cost` SSE chunk after the usage chunk:

```json
{choices: [], x-opencode-type: inference-cost, cost: 0.00006972, normalizedUsage: {...}}
```

(the `cost` field is a string; some gateways send a number). Since `StreamingCompletionChunk` ignores unknown fields, the value is lost and callers cannot report real spend — they must either re-derive cost from token counts (approximate, requires per-model price tables that drift) or not report it.

## Proposal

- Parse the top-level `cost` field (string or number) in `StreamingCompletionChunk`.
- Thread it through `CompatibleChunk` → the streaming loop → `build_final_response`, mirroring how `usage` already flows.
- Surface it on `StreamingCompletionResponse.cost` and propagate it into `completion::Usage::cost` in `token_usage()`, so agent runs and telemetry see it without extra plumbing.

OpenRouter already models provider-reported cost (`usage.cost`); this extends the same capability to OpenAI-compatible streams where the field is a sibling of usage rather than nested inside it.

No breaking changes: the new field defaults to `None` and existing deserialization is unaffected.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Look at the `StreamingCompletionChunk` struct and its deserialization. Follow the flow of the `usage` field through `CompatibleChunk` and `build_final_response` to understand the plumbing. The goal is to add a `cost` field (Option or similar) in the same path, ensuring it ends up in `StreamingCompletionResponse.cost` and `completion::Usage::cost`. Test by simulating a stream with an `inference-cost` chunk.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
rust
Ambito
backend-api-design
Tipo di issue
Funzionalità
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Tranquilla
Chiarezza
Specificata chiaramente
Idoneità per principianti
65/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.