Hmbown / Hmbown/Codewhale

Extract Responses dialect policy behind a language-neutral conformance harness

Open
#5,093 1 comment 1 reaction 2 assignees Claimed by @Hmbown View on GitHub
agent-ready enhancement reliability responses-api tui
Dominant language
Rust
Stars
41k
Forks
3.6k
Avg merge
13h 59m
Merged PRs (30d)
299

Description

## Problem

The Responses bridge already covers substantial request construction, streaming, tool calls, and usage handling, but provider-specific branches are embedded in the adapter and exercised as individual cases. Adding custom Responses routes without a table-driven dialect contract would make regressions likely in tool loops, reasoning, caching, and normalized usage.

v0.9.4 needs a typed dialect-policy seam backed by deterministic conformance fixtures. Both built-in and custom Responses routes should share one normalized runtime contract while declaring their wire differences explicitly.

## Current evidence

- `crates/tui/src/client/responses.rs` owns Responses request construction, SSE event parsing, function-call assembly, and usage extraction.
- `build_responses_body_for_provider` currently receives an `ApiProvider` and contains provider-specific request branches.
- `crates/tui/src/client.rs::prepare_outbound_request` is the shared no-I/O preparation seam used by execution and request preview.
- `crates/tui/src/client/prepared.rs` provides a typed prepared-request boundary suitable for snapshot and contract tests.
- #3019 and #3084 established the reliability and provider-adapter baselines that this work should preserve.

## Scope

1. Introduce an internal typed `ResponsesDialectProfile` (name may vary) that owns declared differences in:
- endpoint and header policy
- stateful versus stateless turns
- reasoning field shape
- supported sampling and output-token fields
- `store` and `include` behavior
- input replay requirements
- supported server-side capabilities
2. Refactor request preparation to consume the profile instead of scattering provider identity checks through body construction. Provider identity may select a profile; it must not implicitly grant unrelated capabilities.
3. Add a table-driven conformance harness covering at least the standard and stateless/plain-reasoning profiles.
4. Cover outbound request fixtures for:
- instructions/system input and exact model identity
- function tools and schema sanitation
- parallel tool-call policy
- function-call outputs and multi-turn replay
- reasoning configuration and reasoning summaries
- stateful/stateless `store` and `include` behavior
5. Cover SSE fixtures for:
- text and reasoning deltas
- split function-call arguments
- completion and usage events
- unknown, malformed, duplicated, and out-of-order events
- UTF-8 boundaries, idle timeout, retry, and terminal errors
6. Verify normalized usage and cache accounting for input, output, reasoning, cache hits, cache misses, and cache writes. Do not fabricate cost when a route has no known SKU.
7. Add a deterministic error matrix:
- invalid profile/configuration fails before network
- 400/401/403 errors fail fast with sanitized diagnostics
- 429/5xx behavior follows the declared retry policy
8. Document how to add a new typed profile and its fixtures.

Tests must use loopback or in-process fixtures only; no live provider traffic.

## Key files

- `crates/tui/src/client/responses.rs`
- `crates/tui/src/client.rs`
- `crates/tui/src/client/prepared.rs`
- `crates/tui/src/models.rs`
- `crates/tui/src/client/cost_status.rs`
- Existing cache-guard and provider-native-search tests where capability gating intersects the profile

## Acceptance criteria

- [ ] Standard and stateless/plain-reasoning profiles pass the same normalized transcript and tool-loop contract wherever the capability is supported.
- [ ] Unsupported features fail before network access with an error naming the selected profile and capability.
- [ ] Production execution and request preview use the same profile-derived prepared request.
- [ ] Function-call argument fragments assemble correctly across event boundaries and never emit a partial tool call.
- [ ] Unknown or malformed events produce deterministic, sanitized behavior without hiding terminal provider errors.
- [ ] Reasoning, token, and cache usage survive normalization without double counting.
- [ ] Cost remains unknown unless the exact route/model SKU is registered.
- [ ] Fixture snapshots contain no credentials or account identifiers.
- [ ] The full harness runs offline and documents how another profile is added.

## Verification

- `cargo fmt --all -- --check`
- `cargo test -p codewhale-tui client::responses`
- `cargo test -p codewhale-tui client::prepared`
- `cargo test -p codewhale-tui cache_guard`
- `cargo clippy -p codewhale-tui --all-targets -- -D warnings`
- Confirm all fixtures are deterministic and require no provider credentials.

## Out of scope

- Live provider certification
- A user-programmable adapter language
- New provider authentication flows
- Default enablement of server-side tools
- Pricing additions without a verified SKU

## Related

- Parent architecture issue: #5092
- Responses reliability baseline: #3019
- Provider descriptor and adapter baseline: #3084

## Triage note

Milestone: **v0.9.4**. This is the adapter-extraction and offline proof slice for the parent architecture issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.