abetlen / abetlen/llama-cpp-python
Capture the stop token upon termination
Abierto
- Lenguaje dominante
- Python
- Estrellas
- 10.6k
- Forks
- 1.4k
- Métricas de merge de PR
- Métricas de PR pendientes
Descripción
**Is your feature request related to a problem? Please describe.**
I have a list of stop tokens `["FOO", "BAR"]` and I would like to know which one triggered the halting mechanism.
**Describe the solution you'd like**
I want to see the corresponding token in the response object, on top of `reason: stop`/
**Describe alternatives you've considered**
Until now I have to increment max_tokens incrementally while the stop token is not spotted in the response.
**Additional context**
Add any other context or screenshots about the feature request here.
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.