apache / apache/shenyu

[BUG] AiProxy stream retry replays the whole upstream, duplicating SSE content

Open
#6,647 1 comment 0 reactions 0 assignees View on GitHub
feature: plugin priority: high type: bug
Dominant language
Java
Stars
8.8k
Forks
3.1k
Avg merge
7d 1h
Merged PRs (30d)
85

Description

- severity: High
- files: `shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/src/main/java/org/apache/shenyu/plugin/ai/proxy/enhanced/service/AiProxyExecutorService.java:88-115`; consumption at `AiProxyPlugin.java:171-182`
- description: `executeStream` wraps `doChatStream` in `Retry.max(1)` then `.onErrorResume(NonTransientAiException.class, handleFallbackStream)`. When the upstream stream errors AFTER emitting some chunks, Reactor `retryWhen` re-subscribes, issuing a brand-new upstream call emitting from the beginning. Those first-attempt chunks were already written to the client SSE response, so the client receives `[partial attempt-1][full attempt-2]`.
- impact: Corrupted/duplicated AI completions for streaming calls that hit a mid-stream error; non-idempotent streaming calls are replayed.
- suggested_fix: For streaming, do not blanket-retry mid-stream; retry only before any emission (buffer-first) or fail and let the client reconnect.
- confidence: High
- related_existing: none — #6575/#6583 concern the generic retry plugin, not the AI-proxy `executeStream` Flux replay.

---
_Identified during the 2026-08-02 deep re-scan; full list in [`docs/scan2-2026-08-02/00-consolidated-critical-high.md`](docs/scan2-2026-08-02/00-consolidated-critical-high.md)._

Contributor guide

No contributing guide indexed for this repository

Research direction

Read executeStream in shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-proxy/src/main/java/org/apache/shenyu/plugin/ai/proxy/enhanced/service/AiProxyExecutorService.java:88-115 and the stream consumption in AiProxyPlugin.java:171-182. Trace the retry behavior after partial emission, then exercise a stream that errors mid-response. Done means the client no longer receives duplicated SSE content from replayed upstream output.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.