abetlen / abetlen/llama-cpp-python
Capture the stop token upon termination
Aperta
- Lingua principale
- Python
- Stelle
- 10.6k
- Fork
- 1.4k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
**Is your feature request related to a problem? Please describe.**
I have a list of stop tokens `["FOO", "BAR"]` and I would like to know which one triggered the halting mechanism.
**Describe the solution you'd like**
I want to see the corresponding token in the response object, on top of `reason: stop`/
**Describe alternatives you've considered**
Until now I have to increment max_tokens incrementally while the stop token is not spotted in the response.
**Additional context**
Add any other context or screenshots about the feature request here.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.