Make response caching safe for Anthropic streaming compaction blocks
@mnajafian-nv is already working on this.
Since Sep 19, 2026.
- Dominant language
- Rust
- Stars
- 173
- Forks
- 73
- Avg merge
- 7h 48m
- Merged PRs (30d)
- 269
Description
Scope
This only affects Relay's optional response cache when an Anthropic request enables server-side compaction or carries a prior compaction block. Ordinary Anthropic traffic is out of scope, and the first live SSE stream is forwarded correctly.
The data-loss edge case occurs when a streamed request actually triggers compaction: cached replay cannot currently reproduce the native compaction_delta lifecycle. Related cache keys also omit the Anthropic beta/version headers, and replay must preserve missing versus explicit-null fields.
#1035 fixes the separate stream-aggregation bug.
Resolution
Until native replay and header-aware keys are implemented, bypass response caching for compaction-enabled and continuation requests. Partial, malformed, or legacy compaction entries must always miss and run live.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.