Hmbown / Hmbown/Codewhale

Make Responses API behavior provider-profiled instead of provider-hard-coded

Open
#5,092 1 comment 1 reaction 2 assignees Claimed by @Hmbown View on GitHub
agent-ready enhancement model-lab reliability responses-api tui
Dominant language
Rust
Stars
41k
Forks
3.6k
Avg merge
13h 59m
Merged PRs (30d)
299

Description

## Problem

Codewhale can resolve `WireFormat::Responses` from a provider/model offering, but the Responses serializer still chooses important wire behavior from provider-enum branches. That makes a new Responses-compatible route more than a base-URL change: request fields, endpoint shape, reasoning representation, replay/state behavior, tool events, headers, and usage normalization can drift independently.

For v0.9.4, Responses behavior should be selected through a typed provider/dialect profile attached to the resolved route. The route resolver still chooses the provider and model; the profile only describes how that exact Responses endpoint speaks.

## Current evidence

- `crates/config/src/provider.rs::WireFormat` and `WirePolicy` already separate Chat Completions, Responses, and Anthropic Messages at the route layer.
- `crates/config/src/provider.rs::Deepseek::wire_policy` is model-aware, while `Custom::wire_policy` is fixed to Chat Completions.
- `crates/tui/src/client/responses.rs::build_responses_body_for_provider` derives an `is_deepseek` boolean and branches on request fields, reasoning, and replay behavior.
- `crates/tui/src/client.rs::prepare_outbound_request` selects Responses body construction, endpoint URL, and route shape using `ApiProvider` checks.
- `crates/tui/src/client/prepared.rs` already provides the typed prepared-request seam that production dispatch and request preview share.
- Closed #185 established that wire format comes from the exact route offering, not a model slug. Closed #3084 established the provider-descriptor/wire-adapter ownership boundary.

## Architecture contract

1. A resolved route carries both `WireFormat::Responses` and a typed Responses dialect/profile.
2. The dialect owns endpoint-path policy, supported request fields, reasoning encoding/replay, tool item/event variants, statefulness, provider-only headers, and usage/cache extraction.
3. `prepare_outbound_request` remains the single no-I/O seam used by production and `/preview-request`.
4. Provider/model selection remains in route resolution. The Responses adapter must not infer a dialect from a model slug or rewrite the selected provider.
5. Profiles are typed and allowlisted. Do not add arbitrary JSON body patches, arbitrary event remapping, or a prompt hook.
6. Existing OpenAI API, ChatGPT OAuth/Codex, DeepSeek, and model-aware gateway behavior must remain byte/fixture compatible unless a fixture demonstrates an existing bug.

## Implementation issues

- [ ] #5093 — extract the typed Responses dialect policy and prove it with an offline conformance harness.
- [ ] #5094 — let named custom providers explicitly select Responses plus an allowlisted dialect profile while preserving the Chat Completions default.

Together these two issues implement the architecture contract above; neither should add model-name or base-URL inference.

## Key files

- `crates/config/src/provider.rs`
- `crates/config/src/provider_kind.rs`
- `crates/config/src/route/resolver.rs`
- `crates/tui/src/client.rs`
- `crates/tui/src/client/prepared.rs`
- `crates/tui/src/client/responses.rs`
- `docs/CONFIGURATION.md`

## Acceptance criteria

- [ ] No Responses request-body or endpoint decision depends on a loose `is_deepseek`/`is_codex` boolean when the same fact can come from the typed dialect profile.
- [ ] Exact route resolution selects wire format and dialect before request serialization.
- [ ] Current OpenAI API, ChatGPT OAuth/Codex, DeepSeek, and model-aware gateway fixtures still pass.
- [ ] One named custom provider can select Responses explicitly without model-name inference.
- [ ] Request preview and production dispatch produce the same normalized prepared request.
- [ ] Unsupported profile/feature combinations fail before network I/O with a provider-named corrective error.
- [ ] All verification is provider-free and uses redacted loopback fixtures; no live credential is required.

## Verification

cargo fmt --all -- --check
cargo test -p codewhale-config --locked
cargo test -p codewhale-tui --bin codewhale-tui --locked client::responses
cargo test -p codewhale-tui --bin codewhale-tui --locked prepared
python3 scripts/check-provider-registry.py
git diff --check

## Out of scope

- Adding or privileging a new built-in provider or model.
- A hosted API proxy or Codewhale-operated inference service.
- Arbitrary user-authored request/response transformation code.
- Pricing claims without an exact provider/model SKU source.
- Live-provider, OAuth, billing, publishing, or release actions.

## Related

- #185 — exact route offerings choose Responses versus Chat Completions; this issue begins after that selection.
- #3019 — existing Responses reliability and tool-result behavior that must not regress.
- #3084 — provider descriptors and wire-adapter ownership contract.
- #5084 — current named-custom-provider compatibility spike; it remains Chat Completions unless this work explicitly enables Responses.

_Triage note: filed for the v0.9.4 milestone on 2026-08-02 from verified current source anchors._

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.