abetlen / abetlen/llama-cpp-python

low level examples broken after [feat: Update sampling API for llama.cpp (#1742)]

Abierto
#1,803 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

I believe that after the commit for "Update sampling API for llama.cpp (#1742)" the low level examples broke.

```
Traceback (most recent call last):
File "/home/jwylie/dev/LLMExplorer/llama_cpp/examples/low_level_api/low_level_api_chat_cpp.py", line 761, in
m.interact()
File "/home/jwylie/dev/LLMExplorer/llama_cpp/examples/low_level_api/low_level_api_chat_cpp.py", line 697, in interact
for i in self.output():
^^^^^^^^^^^^^
File "/home/jwylie/dev/LLMExplorer/llama_cpp/examples/low_level_api/low_level_api_chat_cpp.py", line 664, in output
cur_char = self.token_to_str(id)
^^^^^^^^^^^^^^^^^^^^^
File "/home/jwylie/dev/LLMExplorer/llama_cpp/examples/low_level_api/low_level_api_chat_cpp.py", line 638, in token_to_str
n = llama_cpp.llama_token_to_piece(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: this function takes at least 6 arguments (4 given)
```
The class LlamaSamplingContext is still there, but no longer used, in fact it cannot be used because sample_repetition_penalties was removed. Also, a request, an example of how to get sorted candidate data after a sample would be appreciated, I had a fork using LlamaSamplingContext to store and return candidate data, but that no longer seems to be possible, or at least its no longer clear to me how I might do something similar.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.