abetlen / abetlen/llama-cpp-python

OSError: exception: access violation reading 0xFFFFFFFFFFFFFFFF

Abierto
#666 1 comentario 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

I'm trying to implement the low level API into my own program, loading the model(I am using Pygmalion-13B.ggmlv3.Q6_K.gguf) works fine and I get no errors. Now when I try to evaluate the model via llama_cpp.llama_eval I get:

```
llama_cpp.llama_eval(self.context, (llama_cpp.c_int * len(embd))(*embd), llama_cpp.c_int(len(embd)), llama_cpp.c_int(0), self.NTHREADS)
File "C:\Users\name\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_cpp\llama_cpp.py", line 788, in llama_eval
return _lib.llama_eval(ctx, tokens, n_tokens, n_past, n_threads)
OSError: exception: access violation reading 0xFFFFFFFFFFFFFFFF
```

I've tried type casting the parameters to C integers as seen in the error log snipped above and plain integers too. The self.context is of type llama_cpp.llama_context_p and self.NTHREADS is being retrieved via multiprocessing as seen inside the low level API example in this repository.

My only guess is it's a problem with the model itself, unfortunately I don't have another gguf model at hand which could be used to test this theory.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.