diagnostics: retain bounded structural summaries for successful and incomplete model streams
@jhrozek is already working on this.
Since Sep 18, 2026.
- Dominant language
- Go
- Stars
- 152
- Forks
- 16
- Avg merge
- 14h 48m
- Merged PRs (30d)
- 536
Description
Summary
Mecatl retains content-free request manifests and sanitized evidence for failed or suppressed provider attempts. It deliberately retains no structural evidence for a successful HTTP/model stream.
That leaves a gap for reports such as #1639:
- the provider returns HTTP 200;
- the stream appears to finish;
- the normalized assistant text is nonsensical; and
- there is no durable evidence of whether Mecatl saw normal framing, a provider terminal event, an incomplete stream, or parser/decoder anomalies.
Add target-bound, durable, content-free structural stream evidence. Do not capture raw provider traffic.
Tracking: #1639
This issue improves future diagnosis of reports like #1639. It does not close, resolve, or claim to reproduce #1639.
Current behavior
network.attempt records failed or suppressed provider attempts. ADR 0255 explicitly excludes successful-attempt timing and protocol-phase evidence.
For a clean HTTP 200 stream that produces anomalous text, the existing failed-attempt path may contain no evidence at all.
Proposed behavior
For each provider attempt, retain a bounded structural summary that can answer questions such as:
HTTP status: 200
response framing: SSE
provider terminal frame observed: yes
protocol events accepted: 73
response bytes read: 14,220
local stream outcome: complete
The exact field names and closed vocabularies belong in the acceptance plan and decision record. The design must support at least these outcomes:
- complete;
- incomplete;
- stream error;
- cancelled; and
- evidence unavailable.
The debugger must describe these as observations, not causal conclusions.
For example:
Mecatl observed SSE framing and a provider terminal frame.
is permitted.
The gateway caused the bad response.
is not a conclusion this evidence can support on its own.
Required architecture constraints
- Provider adapters observe provider-specific stream structure.
- A provider-neutral, run-local observer carries only validated structural facts to the agent loop.
- The agent loop remains the sole event producer.
- The server relay remains the sole durable EventLog writer.
- Evidence is target-bound and available only through the session debugger.
- Ordinary gRPC, HTTP/SSE, Team, and event-log-readback streams do not expose it.
- Retried provider attempts remain separately correlated to the same logical turn.
- Missing support is explicit as unavailable; it is never presented as a healthy stream.
- The implementation must preserve streaming behavior: no
ReadAll, no response buffering before delivery, and no change to TTFT, retry, cancellation, or stream-idle semantics.
Strict privacy boundary
This issue must never retain or expose:
- request, prompt, message, tool-schema, argument, or tool-result contents;
- response text or response fragments;
- raw errors or provider error codes;
- URLs, query strings, headers, cookies, tokens, credentials, or environment values;
- provider request/correlation IDs; or
- response-content hashes that become an offline content oracle.
This issue must not add --capture-wire, a capture directory, a raw-response debug mode, or a plaintext diagnostic artifact.
Why this is separate from request-manifest evidence
The request manifest answers:
What content-free request shape did Mecatl prepare?
This issue answers:
What content-free stream structure did Mecatl observe afterward?
A normal manifest cannot show whether the gateway ended an SSE stream correctly. A normal stream summary cannot show how many tool-schema bytes were advertised. The two records diagnose different parts of one turn.
Decision record
Architectural. ADR 0255 explicitly says successful attempts and protocol-phase timings are not measured. This issue requires a new or superseding decision record that defines the new retained-evidence boundary, its vocabulary, validation, visibility, and retention semantics.
Acceptance criteria
- A completed provider attempt produces one bounded, validated structural summary.
- A truncated or errored stream records incomplete/error structure without fabricating a normal terminal frame.
- Retry attempts remain distinguishable and correlate to their logical turn.
- Invalid provider-produced evidence is rejected and never persisted.
- All supported in-tree production provider families have offline fixture coverage.
- Durable relay persistence survives a disconnected client without making a diagnostic persistence failure fail inference.
- Ordinary client streams and public event-log readback omit this evidence.
- The target-bound debugger exposes bounded summaries and explicitly reports unavailable, incomplete, and truncated evidence.
- Debugger wording reports observations only and does not assign fault to Mecatl, a gateway, or a model.
- Engine API compatibility artifacts, changelog, documentation, and the superseding decision record are updated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.