perf(proxy): skeleton request parsing for large bodies (inline base64 multimodal payloads)

Context

Split out of the perf program (AISIX-Cloud#1259, same-protocol passthrough gate). That gate measured, on the benchmark payload, a full-delete upper bound of 3.2–4.6% CPU (≈3.6–5.1 µs/req at c=128) for everything serde/render on the chat path — a correct no-go at that payload size (86 bytes).

The no-go does not generalize to large bodies. Serde cost scales roughly linearly with body size while the rest of the request path is roughly constant. The realistic worst case is multimodal chat: images embedded as base64 data URIs inside messages[].content, with clients resending the whole conversation history every turn — request bodies of hundreds of KB to MB. Today every such request pays, on /v1/chat/completions:

• full typed deserialization of the entire body (axum Json<ChatFormat>, chat.rs:100)
• an unconditional typed → serde_json::Value full-tree conversion (prepare_outbound_body, openai bridge.rs:314) — even when the target has no body overrides
• full re-serialization on the way out (bridge.rs:410)

Three O(body) passes plus a heap allocation for every content string, per request, for bytes the gateway never reads. Peak memory per request is a small multiple of body size.

Proposal — skeleton parsing

Really parse only the fields the gateway reads or rewrites; keep bulk subtrees as raw bytes (serde_json::value::RawValue) and emit them verbatim.

Request side (no user-visible behavior change — the upstream receives the same bytes):

• really parse: model (read + rewrite), stream, stream_options
• keep raw: messages, tools (the two size dominants)
• gate: applies only when no content-consuming feature is enabled for the request (guardrails / PII redaction / semantic cache / token estimation fall back to full parse)

/v1/messages is a simpler first target: its request side is already a Value round-trip whose only rewrite is model (messages.rs:1005).

The response side is a separate decision: envelope fields would stay parsed (model restamp per AISIX-Cloud#410, created, usage extraction) with choices kept raw — but today's render emits only choices[0] and drops unmodeled fields, so raw response passthrough is a user-visible behavior change (n>1 choices and unknown fields would start passing through), not a pure optimization.

Known constraints

• The chat request model keeps unknown fields via #[serde(flatten)]; serde's flatten is incompatible with RawValue, so this needs a hand-rolled partial deserializer or a reshaped extractor.
• Streaming chunks are small; skeleton parsing buys little on the SSE path. The win is the request side and the non-streaming response envelope.

Gate / definition of done

1. Payload-size sweep first, no implementation before it: same-session anchors at c=128 with base64-image-shaped messages at e.g. 1 KB / 8 KB / 64 KB / 512 KB; publish the serde-share growth curve and the crossover size here.
2. Survey how at least three mainstream gateways handle large-body same-protocol forwarding, and cite the serde RawValue documentation for the mechanism.
3. Go: request-side skeleton for the chat family plus /v1/messages, wire-equivalent for the default posture, with tests proving byte-for-byte upstream request equivalence. No-go: record the sweep numbers here and close.

Not scheduled — backlog until a workload with large multimodal payloads makes it worth pulling forward, or the current perf program items land.

Start with the payload-size sweep at c=128 using the stated 1 KB, 8 KB, 64 KB, and 512 KB message sizes, then inspect chat.rs:100, bridge.rs:314 and 410, and messages.rs:1005. Review serde RawValue documentation and survey three mainstream gateways before choosing an approach. Done means a justified go/no-go, or a wire-equivalent chat and /v1/messages implementation with byte-for-byte request tests.