A streaming text delta that repeats everything streamed so far is missing from the recorded turn
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 176
- Forks
- 28
- Avg merge
- 18h 42m
- Merged PRs (30d)
- 16
Description
If a streamed response contains a text delta identical to everything streamed before it, that delta does not make it onto the assistant turn. The caller watches the right text go by, and the turn that gets stored, and sent back as history on the next request, is short by one delta.
The easy case is a response starting with two identical deltas, which is what a leading "\n\n" often looks like coming off a local model.
No API key needed for this, it drives the completions provider's streaming contract:
from openai.types.chat import ChatCompletionChunk
from chatlas._provider_openai_completions import OpenAICompletionsProvider
provider = OpenAICompletionsProvider(api_key="unused", model="gpt-4o-mini")
def chunk(text, last=False):
return ChatCompletionChunk.model_validate({
"id": "chatcmpl-1",
"object": "chat.completion.chunk",
"created": 1,
"model": "gpt-4o-mini",
"choices": [{
"index": 0,
"delta": {"role": "assistant", "content": text},
"finish_reason": "stop" if last else None,
}],
})
deltas = ["\n", "\n", "Here is the answer."]
completion = None
for i, d in enumerate(deltas):
completion = provider.stream_merge_chunks(completion, chunk(d, last=i == len(deltas) - 1))
turn = provider.stream_turn(completion, has_data_model=False)
print("streamed:", repr("".join(deltas)))
print("turn: ", repr(turn.text))
streamed: '\n\nHere is the answer.'
turn: '\nHere is the answer.'
I expected both to be '\n\nHere is the answer.'.
It is not just the first two deltas. ["ab", "c", "abc"] streams abcabc and records abc, while ["ab", "c", "ab"] comes out fine, so it seems to bite when the new delta equals the whole accumulation so far.
chatlas at 7b3aa569, openai 3.13.0, python 3.12.14. Everything built on OpenAICompletionsProvider shares that call, so ChatOllama(), ChatGroq(), ChatDeepSeek() and ChatOpenRouter() are in the same boat. I found it driving the provider rather than a live model, so I cannot say how often a real stream lands on it.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OpenAICompletionsProvider.stream_merge_chunks and stream_turn, using the supplied ChatCompletionChunk reproduction to trace how repeated deltas are accumulated and recorded. Add a regression test covering ["\n", "\n", "Here is the answer."] and the other examples, then verify that the recorded turn matches the complete streamed text.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100