allenai / allenai/asta-autodiscovery

Key the usage `by_model` breakdown by the litellm-qualified model name

Đang mở
#87 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
10
Fork
2
Merge trung bình
1 ngày 21 giờ
Pull request đã merge (30 ngày)
11

Mô tả

Split out of #70 / #86, which deliberately left this alone.

`UsageTracker.record_response` labels each event with `response.model`, the bare name the provider echoes back — `gpt-4o`, not `openai/gpt-4o`. So the `by_model` breakdown in `llm_usage_summary.json`, and the dashboard rows built from it, merge `openai/gpt-4o` and `github_copilot/gpt-4o` into one `gpt-4o` row.

This is no longer a *mispricing*: after #86 the cost of each call is recorded at the source against the fully-qualified model name, and `by_model` sums those recorded costs rather than pricing the label. So the numbers are right; only the row label is ambiguous. That ambiguity is still real for anyone reading a mixed-provider run — and mixed-provider runs are exactly what #68 made expressible.

The fix is small (qualify the label at record time, keeping the provider from the request and the resolved model from the response, e.g. `vertex_ai/gemini-2.5-pro`), but it changes the aggregate keys: every existing model row splits in two across the old/new boundary, and the dashboard shows both forms until pre-change runs age out. That transition is worth deciding on deliberately rather than absorbing as a side effect of the costing change, which is why it is not in #86.

Suggested-by: @dirkraft

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.