google / google/capsem

[triage:tooling-06-missing-tests-mock-http-routing-shapes] Mock server HTTP routing and response shapes are almost entirely untested

Open
#150 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Rust
Stars
72
Forks
13
Avg merge
1d 2h
Merged PRs (30d)
5

Description

Imported from Capsem triage report `tooling-06-missing-tests-mock-http-routing-shapes.md`.

- Severity: `medium`
- Category: `missing-test`
- Area: `capsem-mock-server`
- Location: `crates/capsem-mock-server/src/main.rs:306-602` (`route`), MCP/model builders `745-1317`; tests in `mod tests` at `1631-1677`
- Confidence: `verified`

## Summary
The mock exists to stand in for real upstreams so proxy/model/MCP tests can assert against a known shape. Yet only three things are tested: `deterministic_bytes` caching, the DNS fixture answer, and the WebSocket accept hash. The entire HTTP `route` table — the tool-call branching in `openai_chat_response`, the streaming-vs-nonstreaming branch selection, `mcp_response`, `responses_response`, the Anthropic/Gemini shapes, and the `log_request` record schema — has no coverage. A drift between the mock's emitted shape and the contract the proxy tests rely on would not be caught here.

## Evidence
Test module (lines 1631-1677) contains exactly: `deterministic_bytes_are_cached_and_correct` (1636), `dns_fixture_answers_known_names_and_rejects_unknown` (1645), `websocket_accept_matches_rfc_fixture` (1659). No test constructs a request and calls `route()`, nor asserts on any model/MCP builder output.

High-value untested logic with real branching:
- `openai_chat_response` (745): emits a tool call iff `tools` present OR `is_baked_doctor_openai_smoke` matches (793) — a narrow exact-match heuristic (single user message, content == "hello", model == "mock-local") that silently changes the response shape.
- `mcp_response` (1256): method dispatch for initialize/tools.list/tools.call/resources.* and the `-32601` fallback.
- `mcp_payload_should_delay` (1319): the `doc://slow` / `slow_sleep` trigger that injects a 2s sleep.
- `find_hex32` (1220) / `find_root_txt_path` (1228) / `write_target` (1212): token/path extraction that determines the `printf ... > path` command every model fixture emits.
- `log_request` (1339): the JSON record schema (lowercased headers, `/log` body omission, response-byte accounting via the `LogBody` extension).

## Impact
The mock's job is contract fidelity. Any of the above can drift (e.g. a builder field rename, a changed dispatch key, a broken token-extraction regex) and every consuming test would keep passing while silently asserting against the wrong upstream shape — exactly the failure mode that makes a mock dangerous.

## Suggested fix
Add unit tests that call `route()` (or the individual builders) with representative request bodies and assert on the parsed JSON shape: tool-call vs no-tool-call selection, MCP dispatch results and the error fallback, `write_target` extraction over a payload containing a hex32 token and a `/root/...txt` path, and a `log_request` round-trip asserting the record fields (especially `/log` body omission and lowercased header keys).

## Triage
Confirmed from the local reviewed report in `/Users/elie/git/capsem/tmp/bugs/tooling-06-missing-tests-mock-http-routing-shapes.md`. Track implementation in the triage sprint; add regression coverage before fixing.

Contributor guide

Open the contributing guide

Research direction

Start in crates/capsem-mock-server/src/main.rs, reading route, the model/MCP builders, and the tests module at lines 1631-1677. Add representative tests for routing shapes, MCP dispatch and fallback, write_target extraction, and log_request records. Done means parsed JSON assertions cover the listed branches, including tool selection, token/path extraction, lowercased headers, and /log body omission.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.