abetlen / abetlen/llama-cpp-python
Last message when streaming completion should not be empty
Open
bug
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.4k
- PR merge metrics
- PR metrics pending
Description
When streaming completions the last message is sometimes empty to ensure that the last message contains a finish_reason. This is correct according to the OpenAI schema but is not expected by all clients and can cause bugs (noticed this in openplayground).
Contributor guide
Assessment
This issue has not been assessed yet.