langgenius / langgenius/dify

fix(tts): long replies time out streaming from plugin daemon (qwen3-tts-flash multi-sentence TTS)

Open
#41,456 4 comments 1 reaction 0 assignees View on GitHub
1.17.0
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

## Self Checks

- I have read the Contributing Guide and Language Policy.
- I have searched for existing issues, including closed ones.
- I confirm that I am using English to submit this report.

## Dify version

1.17.0 (self-hosted, Docker Compose)

## Cloud or Self Hosted

Self Hosted (Docker)

## Steps to reproduce

1. Configure Tongyi `qwen3-tts-flash` as the TTS model (plugin `langgenius/tongyi:0.2.15`, which contains the WAV merge fix from dify-official-plugins#3698).
2. Enable TTS in a chatflow/chat assistant whose typical model replies are long (e.g. 1,500-2,000+ characters, several sentences after the plugin's 512-char sentence split).
3. In the chat web app, click the audio-play button on an assistant message.

The plugin invokes DashScope per sentence (512-char limit) and, with the #3698 implementation, downloads each completed WAV segment serially inside `invoke_remote()` and only then starts yielding merged audio to the API. For a multi-sentence reply the total provider synthesis time easily exceeds the API's stream-read window for plugin responses.

## ✔️ Expected Behavior

Audio for long assistant replies should play (even if it takes a while to synthesize), or the pipeline should stream audio out sentence-by-sentence instead of buffering the whole reply.

## ❌ Actual Behavior

The API aborts the plugin stream with a read timeout and the browser gets no audio at all. Console-side, nothing plays; API log shows:

```
2026-08-28 13:54:40.708 ERROR [...] Stream request to Plugin Daemon Service failed
...
httpcore.ReadTimeout: timed out
```

`core/plugin/impl/base.py::_stream_request` → `httpx` read timeout while awaiting plugin chunks; the plugin-side invoke was still running (seen as a 200 with long latency in plugin daemon logs).

Short texts (single sentence, e.g. the model-config "试听/preview" button) always work, which makes this easy to miss: only multi-sentence replies fail.

## Additional context

- Dify 1.17 core + tongyi plugin 0.2.15 (both containing the #41043 / #3698 fixes). The truncated-WAVE issue from #35880 is resolved; this is the remaining long-text failure mode.
- Related hardening gap in #3698's `merge_wav_segments()`: DashScope `qwen3-tts-flash` streaming returns WAV headers whose RIFF/data sizes are int32 placeholders (0x7FFFFFFF family), so the strict `declared == actual` frame validation always rejects valid audio with "DashScope returned truncated WAVE segment 1". Workaround that fixed it for us: treat sizes >= 0x7F000000 as streaming placeholders and validate against actual payload bytes instead.
- Practical workaround for the timeout: reduce `word_limit` in `models/tts/qwen3-tts-flash.yaml` (512 → 100), so each sentence synthesizes in seconds and the whole chain returns well within the timeout.
- Suggested fixes (either/both):
1. Stream audio out per-sentence from the plugin (merge incrementally) instead of buffering all segments before yielding.
2. Make the plugin-daemon stream read timeout configurable / longer for TTS invocations, or have the plugin emit a heartbeat chunk while synthesizing.

Happy to provide full logs/tracebacks if useful.

Contributor guide

Open the contributing guide

Research direction

Start with core/plugin/impl/base.py::_stream_request and trace how the plugin response is read while the plugin-side invoke_remote() processes sentence audio. Then inspect merge_wav_segments() and models/tts/qwen3-tts-flash.yaml, using the reported timeout and streaming-placeholder behavior as starting points. Done means long multi-sentence replies produce playable audio without the API read timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.