anomalyco / anomalyco/opencode

Assistant message modelID records the requested model, never reconciled against the provider response

Open
#38,543 1 comment 2 reactions 1 assignee View on GitHub

@nexxeln is already working on this.

Since Jul 23, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Summary

Assistant messages record providerID/modelID from the request-side configuration (input.model.id), and the provider response's own model assertion is never compared against it. When a provider or router substitutes a model (fallbacks, load-shedding, aggregator routing), the session transcript still reports the requested model as the model used.

Where (dev @ 849c2598a)

  • packages/opencode/src/session/llm.ts:88,262,285modelID: input.model.id is stamped into the assistant message at request-construction time.
  • Provider responses (e.g. OpenAI/Anthropic API response bodies) carry a response-asserted model identifier, but it is not captured on the message or checked against the requested value.
  • packages/opencode/src/session/message-v2.ts:245 — downstream logic (differentModel) compares configured model vs message metadata, so today it can only ever compare a request-side claim against another request-side claim.

Why it matters

  • Routers/aggregators substitute models by design — OpenRouter-style fallbacks and gateways can serve a different model than requested; users reading opencode stats, exported sessions, or the session API have no way to see this happened.
  • Cost/token accounting keyed to the wrong model produces wrong cost reporting.
  • Provenance: for anyone using session exports downstream (evals, fine-tuning data, audit), the recorded model is a low-confidence data point today but is presented with the same confidence as everything else. Even without cryptographic attestation, a response-asserted model is a strictly stronger signal than a request-side echo — and a mismatch is a signal worth surfacing on its own.

Proposal

  1. Capture the response-asserted model identifier (where the provider SDK exposes it) onto the assistant message alongside the existing request-side modelID — e.g. modelID (requested) + responseModelID (provider-asserted, nullable when unavailable).
  2. When both exist and disagree, surface it: a warning log and/or a message flag (model_mismatch: true) that the TUI/stats can render.
  3. No behavior change otherwise — purely additive observability.

Happy to discuss shape; the nullable-additive version seems fully backward-compatible with existing session exports.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.