abetlen / abetlen/llama-cpp-python

Capture the stop token upon termination

未關閉
#1,182 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
10.6k
分支
1.4k
PR 合併指標
PR 指標待擷取

描述

**Is your feature request related to a problem? Please describe.**
I have a list of stop tokens `["FOO", "BAR"]` and I would like to know which one triggered the halting mechanism.

**Describe the solution you'd like**
I want to see the corresponding token in the response object, on top of `reason: stop`/

**Describe alternatives you've considered**
Until now I have to increment max_tokens incrementally while the stop token is not spotted in the response.

**Additional context**
Add any other context or screenshots about the feature request here.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。