abetlen / abetlen/llama-cpp-python
Feature request: add support for streaming tool use
オープン
- 主要言語
- Python
- スター
- 10.6k
- フォーク
- 1.4k
- PR マージ指標
- PR 指標を取得中
説明
The combination `stream=True, tool_choice="auto"` raises an exception right now, which means that developers are stuck with one of two unfortunate choices:
1. Developing an application that streams the response but cannot use tools
2. Developing an LLM application that can use tools but cannot stream the response
Relevant discussion: https://github.com/abetlen/llama-cpp-python/discussions/1615
コントリビューションガイド
評価
この issue はまだ評価されていません。