Why is ChatCompletionService#createStreaming blocking the thread for a significant amount of time
Nobody has claimed this yet.
- Dominant language
- Kotlin
- Stars
- 1.5k
- Forks
- 264
- Avg merge
- 9h 46m
- Merged PRs (30d)
- 96
Description
When attempting to stream data with a client like so:
try (var stream = client.chat().completions().createStreaming(params))
the thread is blocked at this line of code for 5-15 seconds after which the response is nearly instant
It feels like the data is getting buffered until the entire response is returned and only then will the stream return
Using ChatModel.GPT_5_NANO
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the delay at ChatCompletionService#createStreaming with ChatModel.GPT_5_NANO and inspect the streaming response path. Verify whether the first chunk arrives promptly rather than after the complete response, and use that timing as the completion criterion; the payload names no source file or test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kotlin
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100