abetlen / abetlen/llama-cpp-python

Capture the stop token upon termination

Abierto
#1,182 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

**Is your feature request related to a problem? Please describe.**
I have a list of stop tokens `["FOO", "BAR"]` and I would like to know which one triggered the halting mechanism.

**Describe the solution you'd like**
I want to see the corresponding token in the response object, on top of `reason: stop`/

**Describe alternatives you've considered**
Until now I have to increment max_tokens incrementally while the stop token is not spotted in the response.

**Additional context**
Add any other context or screenshots about the feature request here.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.