abetlen / abetlen/llama-cpp-python
Can't compute multiple embeddings in a single call
- Lenguaje dominante
- Python
- Estrellas
- 10.6k
- Forks
- 1.4k
- Merge medio
- 5 h 23 min
- PR fusionados (30 d)
- 5
Descripción
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.
# Expected Behavior
Running this code:
```python
model = llama_cpp.Llama ("mxbai-embed-xsmall-v1-q8_0.gguf", embedding = True)
embeddings = model.embed (["Hello", "World"])
```
used to work in v0.3.14
# Current Behavior
The code raises an exception `RuntimeError: llama_decode returned -1`. The following messages are printed to the console:
```
init: invalid seq_id[3][0] = 1 >= 1
encode: failed to initialize batch
```
# Environment and Context
llama-cpp-python was compiled in CUDA mode
# Failure Information (for bugs)
Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.
# Steps to Reproduce
```python
Python 3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import llama_cpp
>>> model = llama_cpp.Llama ("../models/mxbai-embed-xsmall-v1-q8_0.gguf", embedding = True)
...
>>> embeddings = model.embed (["Hello", "World"])
decode: cannot decode batches with this context (calling encode() instead)
init: invalid seq_id[3][0] = 1 >= 1
encode: failed to initialize batch
llama_decode: failed to decode, ret = -1
Traceback (most recent call last):
File "", line 1, in
File ".../site-packages/llama_cpp/llama.py", line 1108, in embed
decode_batch(s_batch)
File ".../site-packages/llama_cpp/llama.py", line 1045, in decode_batch
self._ctx.decode(self._batch)
File ".../site-packages/llama_cpp/_internals.py", line 327, in decode
raise RuntimeError(f"llama_decode returned {return_code}")
RuntimeError: llama_decode returned -1
```
Guía de contribución
Línea de trabajo
El error se produce en llama_cpp/llama.py, alrededor de la línea 1108, en el método embed, y en la línea 1045, en decode_batch. Empieza examinando la inicialización del batch y la gestión de los identificadores de secuencia en los bindings de C++ (_internals.py). Revisa los cambios en la lógica de embeddings entre v0.3.14 y la versión actual. Ejecuta el script de reproducción proporcionado para ver el error exacto y, después, comprueba la decodificación del batch de la biblioteca llama.cpp para múltiples secuencias.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- c, python
- Área
- ai, machine-learning
- Tipo de issue
- Error
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Estado de actividad
- Estancado
- Claridad
- Bien especificado
- Aptitud para principiantes
- 40/100