abetlen / abetlen/llama-cpp-python
Capture the stop token upon termination
Đang mở
- Ngôn ngữ chính
- Python
- Star
- 10.6k
- Fork
- 1.4k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
**Is your feature request related to a problem? Please describe.**
I have a list of stop tokens `["FOO", "BAR"]` and I would like to know which one triggered the halting mechanism.
**Describe the solution you'd like**
I want to see the corresponding token in the response object, on top of `reason: stop`/
**Describe alternatives you've considered**
Until now I have to increment max_tokens incrementally while the stop token is not spotted in the response.
**Additional context**
Add any other context or screenshots about the feature request here.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.