Affinity carriers this instance cannot authenticate are forwarded upstream and dissolve the routing pin with no attributable record
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 85
- Forks
- 33
- Avg merge
- 45m
- Merged PRs (30d)
- 10
Description
Provenance. This report was written by Claude Code. A human operator asked me to
investigate a suspected defect, and this is my own summary of the mechanism, the
reproduction, the impact, and a fix direction — not a hand-authored bug report. I have
deliberately filed an issue rather than a PR, because the fix involves a design choice I
do not think an outside agent should make unilaterally. Please run your own Claude Code
over the claims below to verify and amend them; I have tried to mark precisely where I am
confident and where I am not. Line references are against7c1e79f9.
Problem
AffinityCodec.unwrap returns { kind: 'foreign', value } for two situations that are not
alike:
- the value carries no Floway framing — genuinely someone else's blob, and forwarding it
verbatim is correct; - the value carries Floway's framing but this instance cannot authenticate it — a carrier
this deployment plausibly issued, under a key or a domain that no longer matches.
The fail-open itself is right and I am not asking for it to change: a carrier this instance
cannot open may belong to a Floway it is chained behind, and stripping the trailer would
corrupt that case. codec_test.ts pins this deliberately.
The problem is that case (2) has three silent consequences and leaves no trace anywhere in
Floway's own surfaces, so an operator debugging it has nothing to look at.
Evidence
unwrap computes the discriminating signal and discards it. splitOpaqueTrailer
succeeding is what separates (1) from (2):
// packages/gateway/src/data-plane/chat/shared/affinity/carrier.ts:136-137
const framed = splitOpaqueTrailer(value, IV_BYTES + 16);
if (framed === null) return { kind: 'foreign', value }; // (1) no framing
// …
// :159-161 — (2) framed like ours, failed to authenticate
} catch { return { kind: 'foreign', value }; }
Downstream, both collapse to the same verdict and the value is written into the
upstream-bound payload:
// packages/gateway/src/data-plane/chat/shared/affinity/selection.ts:50 and :60
if (decoded.kind === 'foreign') return { kind: 'preserve', value: decoded.value };
// packages/gateway/src/data-plane/chat/responses/affinity/ingress.ts:145
if (projection.kind === 'preserve') replacement[location.slot] = projection.value;
Three separate behaviours hang off the single kind === 'owned' gate, and all three fail
open together:
// ingress.ts:108-110 — only owned blobs pin routing
if (location.decoded.kind === 'owned') { latestOwnedTarget = location.decoded.affinity;
if (required) requiredTargets.push(latestOwnedTarget); }
// ingress.ts:123 — only owned synthetic items are removed before dispatch
synthetic: blobs.some(blob => blob.decoded.kind === 'owned' && blob.decoded.syntheticItem === true),
// selection.ts:123-129 — the routing-unavailable guard is keyed on requiredTargets
With requiredTargets empty, the guard is dead and the consistency assertion at
selection.ts:81-84 is vacuously satisfied.
Nothing records the event: there is no log, metric or counter anywhere under
data-plane/chat/shared/affinity/ or data-plane/chat/responses/affinity/. Dumps do not
help either — respond.ts:56-57 runs observeResponsesFrames before
wrapResponsesClientEgress, so the dump holds pre-wrap response frames, and DumpRecord
(packages/gateway/src/dump/types.ts) has no upstream-request record at all. There is no
stored artifact of what Floway handed either party.
Reachability
This is the part I most want checked, because it is what decides whether this is a defect
or a diagnostic nicety.
The AES-GCM branch is entered whenever the key or the AAD differs from wrap time. The AAD
is domain ‖ originalBytes (carrier.ts:94-98, 126), and the domain is
responses.<item.type>.<slot> (ingress.ts:43-47). So it is reached by ordinary
operational events, not only by tampering:
- a server-secret change — rotation, or a restore through the control-plane
data-transfer path, or two instances that do not share the same secret; - a carrier-domain change — any change to item-type canonicalization, or to which
items are treated as affinity carriers, invalidates every carrier already in a client's
conversation history. Draft PR #273 changes exactly that set, which is part of why I am
raising this now; - a client replaying an older conversation across either of the above.
Clients hold these blobs in conversation history and replay them for a long time, so the
window is not short.
What this issue does not claim
An earlier draft of this analysis argued that two of unwrap's exits are reachable only
after a successful authenticated decrypt and are therefore "provably this gateway's own".
On checking, that is wrong and I am withdrawing it:
carrier.ts:156(originundefined with non-emptyoriginal) is provably
unreachable.wrapemitsoriginexactly when a value was supplied, and binds that
value's bytes into the AAD, soorigin === undefinedimplies the authenticated original
was empty. Reaching it would require an AES-GCM forgery.carrier.ts:152(parseAffinityDatareturning null after a successful decrypt) is not
reachable from production code; it needs a malformedAffinityTargetthat the type
system prevents. It becomes reachable only under version skew —hasOnlyKeys
(carrier.ts:28-29) is a closed check, so a carrier minted by a newer build carrying a
newAffinityDatakey is rejected by an older one after a rollback. That is worth
knowing, but it is a forward-compatibility property, not a current bug.
I also make no claim that this produces the invalid_encrypted_content rejections that
prompted the investigation. Which dimension chatgpt.com binds encrypted reasoning content
to is not publicly documented and I could not establish the link.
Impact
When case (2) occurs, all of the following happen together and none are visible:
- the affinity pin dissolves —
requiredTargetsis empty, so any viable candidate may
serve a turn that was pinned to a specific upstream, and therouting-unavailableguard
never fires; - a synthetic carrier item — 100% Floway ciphertext, zero upstream-origin bytes — survives
the removal atingress.ts:123and is transmitted to a third-party provider; - an ordinary carrier reaches the upstream with Floway's trailer still attached.
The same selection.ts preserve path is shared by the chat-completions, messages and
gemini ingress modules, so this is not Responses-specific.
Fix direction
The behaviour to keep is the pass-through; the thing to recover is attribution. The
smallest change that does that is to carry the reason on DecodedAffinityBlob — still
kind: 'foreign', still byte-identical on the wire — and report it one level up, where
there is a request context to correlate against.
Three constraints I would ask any fix to respect, all of which caught me out:
- The framing gate is not evidence of Floway authorship.
splitOpaqueTraileraccepts
when the trailinguint16belands in[28, N-2], so for an arbitrary base64 blob the
false-positive rate is(min(N-2, 65535) - 27) / 65536— size-dependent, not constant.
That is ~0.7% at 500 bytes, ~6% at 4 KB, ~15% at 10 KB, and ~100% above 64 KB. Reasoning
encrypted_contentsits in the KB range, so reporting per-blob on a framing hit would
misattribute a large fraction of genuinely foreign values. unwrapruns once per blob per item with no bound on item count, and the input is
client-supplied. Any per-occurrence emission is a client-controlled amplifier; this wants
aggregation — one record per request with a count — rather than one line per blob.carrierDomaininterpolates unvalidated client input.canonicalItemType
(ingress.ts:43-44) only remapscompaction_summaryand otherwise passesitem.type
through verbatim, so the domain string can contain arbitrary client-chosen characters
including newlines. It must be escaped and length-bounded before it reaches any log or
record.
A counter on the existing telemetry surface, keyed by reason, may serve better than a log
line — that is the design call I would rather leave to you.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/gateway/src/data-plane/chat/shared/affinity/carrier.ts and trace unwrap into selection.ts and responses/affinity/ingress.ts, then review codec_test.ts for the required pass-through behavior. Confirm how framed authentication failures are distinguished, aggregated per request, and safely reported without changing forwarded bytes; done means the existing fail-open behavior remains covered while the event becomes attributable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100