[BUG] Token-limiter BodyWriter unboundedly buffers the entire (non-gzip) response for the fallback scan
- Dominant language
- Java
- Stars
- 8.8k
- Forks
- 3.1k
- Avg merge
- 7d 1h
- Merged PRs (30d)
- 85
Description
- severity: Medium
- files: `shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-token-limiter/src/main/java/org/apache/shenyu/plugin/ai/token/limiter/AiTokenLimiterPlugin.java:349-392`
- description: `appendResponse` writes every processed chunk into a `BodyWriter` backed by an unbounded `ByteArrayOutputStream`, solely so the `doFinally` fallback can regex-scan the full output if per-chunk parsing missed `completion_tokens`. For a long non-gzip SSE stream this accumulates the entire response in heap with no cap.
- impact: A large streaming AI response can exhaust gateway heap even without gzip.
- suggested_fix: Cap the buffered bytes (ring/last-N-KB buffer); only the final usage chunk is needed.
- confidence: Medium
- related_existing: none — SEC-9 is the gzip Inflater path; this is a distinct OOM vector from unbounded plain-byte accumulation.
---
_Identified during the 2026-08-02 deep re-scan; full list in [`docs/scan2-2026-08-02/06-medium-tiers.md`](docs/scan2-2026-08-02/06-medium-tiers.md)._
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with AiTokenLimiterPlugin.java:349-392, tracing appendResponse, BodyWriter, and the doFinally fallback scan; use docs/scan2-2026-08-02/06-medium-tiers.md for the finding context. Done means long non-gzip SSE responses no longer accumulate without a bound, while the fallback can still detect the final usage data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100