apache / apache/shenyu

[BUG] — AI request/response transformer + MCP decorator: `collectList` on streaming responses breaks backpressure / unbounded in-memory buffering

Open
#6,795 1 comment 0 reactions 0 assignees View on GitHub
feature: plugin priority: medium type: performance
Dominant language
Java
Stars
8.8k
Forks
3.1k
Avg merge
7d 1h
Merged PRs (30d)
85

Description

- severity: Medium
- files: `shenyu-plugin-ai/shenyu-plugin-ai-request-transformer/.../AiRequestTransformerPlugin.java:122-125`, `.../ai-response-transformer/.../AiResponseTransformerPlugin.java:353-356`, `shenyu-plugin-mcp-server/.../response/NonCommittingMcpResponseDecorator.java:85,98`
- description: `chatClient.prompt().stream().content().collectList().map(list -> list.stream()...joining)` and `Flux.from(body).collectList()` subscribe to a (potentially unbounded) streaming `Flux` and buffer all emitted chunks into a `List` in heap before any downstream processing — no `.limitRate`, no chunked pass-through. For large LLM/streaming HTTP responses this defeats streaming backpressure and can buffer the entire response body.
- impact: Heap pressure and added latency for large AI/streaming responses through the gateway.
- suggested_fix: Process chunks incrementally (`concatMap`/`flatMap` per chunk) or add `.limitRate`; stream through rather than aggregating via `collectList`.
- confidence: Medium-High

---
_Identified during the 2026-08-02 deep re-scan; full list in [`docs/scan2-2026-08-02/06-medium-tiers.md`](docs/scan2-2026-08-02/06-medium-tiers.md)._

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the collectList call sites in AiRequestTransformerPlugin.java:122-125, AiResponseTransformerPlugin.java:353-356, and NonCommittingMcpResponseDecorator.java:85,98. Trace how each streaming Flux is consumed and verify that large responses are processed incrementally without buffering the entire stream, while preserving downstream backpressure and response behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.