0xPlaygrounds / 0xPlaygrounds/rig

feat: capture provider-reported cost from OpenAI-compatible streaming responses

Open
#2,242 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
8.6k
Forks
959
Avg merge
4h 32m
Merged PRs (30d)
117

Description

## Problem

Several OpenAI-compatible gateways report the monetary cost of a request in the chat-completions stream, but rig drops it. Notably **OpenCode Go** emits a dedicated `inference-cost` SSE chunk after the usage chunk:

```json
{choices: [], x-opencode-type: inference-cost, cost: 0.00006972, normalizedUsage: {...}}
```

(the `cost` field is a string; some gateways send a number). Since `StreamingCompletionChunk` ignores unknown fields, the value is lost and callers cannot report real spend — they must either re-derive cost from token counts (approximate, requires per-model price tables that drift) or not report it.

## Proposal

- Parse the top-level `cost` field (string or number) in `StreamingCompletionChunk`.
- Thread it through `CompatibleChunk` → the streaming loop → `build_final_response`, mirroring how `usage` already flows.
- Surface it on `StreamingCompletionResponse.cost` and propagate it into `completion::Usage::cost` in `token_usage()`, so agent runs and telemetry see it without extra plumbing.

OpenRouter already models provider-reported cost (`usage.cost`); this extends the same capability to OpenAI-compatible streams where the field is a sibling of usage rather than nested inside it.

No breaking changes: the new field defaults to `None` and existing deserialization is unaffected.

Contributor guide

Open the contributing guide

Research direction

Look at the `StreamingCompletionChunk` struct and its deserialization. Follow the flow of the `usage` field through `CompatibleChunk` and `build_final_response` to understand the plumbing. The goal is to add a `cost` field (Option or similar) in the same path, ensuring it ends up in `StreamingCompletionResponse.cost` and `completion::Usage::cost`. Test by simulating a stream with an `inference-cost` chunk.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.