abetlen / abetlen/llama-cpp-python
Feature request: add support for streaming tool use
Aperta
- Lingua principale
- Python
- Stelle
- 10.6k
- Fork
- 1.4k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
The combination `stream=True, tool_choice="auto"` raises an exception right now, which means that developers are stuck with one of two unfortunate choices:
1. Developing an application that streams the response but cannot use tools
2. Developing an LLM application that can use tools but cannot stream the response
Relevant discussion: https://github.com/abetlen/llama-cpp-python/discussions/1615
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.