abetlen / abetlen/llama-cpp-python
Feature request: add support for streaming tool use
未關閉
- 主要語言
- Python
- 星號
- 10.6k
- 分支
- 1.4k
- PR 合併指標
- PR 指標待擷取
描述
The combination `stream=True, tool_choice="auto"` raises an exception right now, which means that developers are stuck with one of two unfortunate choices:
1. Developing an application that streams the response but cannot use tools
2. Developing an LLM application that can use tools but cannot stream the response
Relevant discussion: https://github.com/abetlen/llama-cpp-python/discussions/1615
貢獻指南
評估
這個 Issue 還沒有評估資料。