apache / apache/shenyu

[BUG] Token-limiter BodyWriter unboundedly buffers the entire (non-gzip) response for the fallback scan

Open
#6,758 1 comment 0 reactions 0 assignees View on GitHub
feature: plugin priority: medium type: bug
Dominant language
Java
Stars
8.8k
Forks
3.1k
Avg merge
7d 1h
Merged PRs (30d)
85

Description

- severity: Medium
- files: `shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-token-limiter/src/main/java/org/apache/shenyu/plugin/ai/token/limiter/AiTokenLimiterPlugin.java:349-392`
- description: `appendResponse` writes every processed chunk into a `BodyWriter` backed by an unbounded `ByteArrayOutputStream`, solely so the `doFinally` fallback can regex-scan the full output if per-chunk parsing missed `completion_tokens`. For a long non-gzip SSE stream this accumulates the entire response in heap with no cap.
- impact: A large streaming AI response can exhaust gateway heap even without gzip.
- suggested_fix: Cap the buffered bytes (ring/last-N-KB buffer); only the final usage chunk is needed.
- confidence: Medium
- related_existing: none — SEC-9 is the gzip Inflater path; this is a distinct OOM vector from unbounded plain-byte accumulation.

---
_Identified during the 2026-08-02 deep re-scan; full list in [`docs/scan2-2026-08-02/06-medium-tiers.md`](docs/scan2-2026-08-02/06-medium-tiers.md)._

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with AiTokenLimiterPlugin.java:349-392, tracing appendResponse, BodyWriter, and the doFinally fallback scan; use docs/scan2-2026-08-02/06-medium-tiers.md for the finding context. Done means long non-gzip SSE responses no longer accumulate without a bound, while the fallback can still detect the final usage data.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.