allenai / allenai/asta-autodiscovery
Key the usage `by_model` breakdown by the litellm-qualified model name
- Ngôn ngữ chính
- Python
- Star
- 10
- Fork
- 2
- Merge trung bình
- 1 ngày 21 giờ
- Pull request đã merge (30 ngày)
- 11
Mô tả
Split out of #70 / #86, which deliberately left this alone.
`UsageTracker.record_response` labels each event with `response.model`, the bare name the provider echoes back — `gpt-4o`, not `openai/gpt-4o`. So the `by_model` breakdown in `llm_usage_summary.json`, and the dashboard rows built from it, merge `openai/gpt-4o` and `github_copilot/gpt-4o` into one `gpt-4o` row.
This is no longer a *mispricing*: after #86 the cost of each call is recorded at the source against the fully-qualified model name, and `by_model` sums those recorded costs rather than pricing the label. So the numbers are right; only the row label is ambiguous. That ambiguity is still real for anyone reading a mixed-provider run — and mixed-provider runs are exactly what #68 made expressible.
The fix is small (qualify the label at record time, keeping the provider from the request and the resolved model from the response, e.g. `vertex_ai/gemini-2.5-pro`), but it changes the aggregate keys: every existing model row splits in two across the old/new boundary, and the dashboard shows both forms until pre-change runs age out. That transition is worth deciding on deliberately rather than absorbing as a side effect of the costing change, which is why it is not in #86.
Suggested-by: @dirkraft
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.