[Feature] Support chat completion stream with tool calls
Open
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
Using chat completion in stream mode with tool calls are becoming more and more popular. While Lmdeploy does support stream and tool, they can't be used at the same time, as Lmdeploy auto sets stream to False when tool is passed and tool_choice is not none.
### Related resources
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.