abetlen / abetlen/llama-cpp-python

Embedding document fails if tokens length > n_ctx

Aberta
#831 0 comentários 0 reações 0 responsáveis Ver no GitHub
bug
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [X ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [ X] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [ X] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [ X] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

Embedding a document, if embedding document tokens are > to n_ctx , it should warn or exit()

# Current Behavior
it fails with error
File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in embed_documents
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 905, in embed
return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 869, in create_embedding
self.eval(tokens)
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 549, in eval
self.input_ids[self.n_tokens : self.n_tokens + n_tokens] = batch
ValueError: could not broadcast input array from shape (8,) into shape (0,)

# Environment and Context
# Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

# Steps to Reproduce

document = open("DOC/mybigdocument.txt,"r")
text = document.read()
embeddings = LlamaCppEmbeddings(model_path=model_path,verbose=False)
embeddings.embed_documents([text])
# Failure Logs

Pl File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in embed_documents
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 905, in embed
return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 869, in create_embedding
self.eval(tokens)
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 549, in eval
self.input_ids[self.n_tokens : self.n_tokens + n_tokens] = batch
ValueError: could not broadcast input array from shape (8,) into shape (0,)

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.