abetlen / abetlen/llama-cpp-python
Capture the stop token upon termination
未關閉
- 主要語言
- Python
- 星號
- 10.6k
- 分支
- 1.4k
- PR 合併指標
- PR 指標待擷取
描述
**Is your feature request related to a problem? Please describe.**
I have a list of stop tokens `["FOO", "BAR"]` and I would like to know which one triggered the halting mechanism.
**Describe the solution you'd like**
I want to see the corresponding token in the response object, on top of `reason: stop`/
**Describe alternatives you've considered**
Until now I have to increment max_tokens incrementally while the stop token is not spotted in the response.
**Additional context**
Add any other context or screenshots about the feature request here.
貢獻指南
評估
這個 Issue 還沒有評估資料。