abetlen / abetlen/llama-cpp-python
Feature request: add support for streaming tool use
Aberta
- Linguagem predominante
- Python
- Estrelas
- 10.6k
- Forks
- 1.4k
- Métricas de merge de PRs
- Métricas de PR pendentes
Descrição
The combination `stream=True, tool_choice="auto"` raises an exception right now, which means that developers are stuck with one of two unfortunate choices:
1. Developing an application that streams the response but cannot use tools
2. Developing an LLM application that can use tools but cannot stream the response
Relevant discussion: https://github.com/abetlen/llama-cpp-python/discussions/1615
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.