abetlen / abetlen/llama-cpp-python

Embedding document fails if tokens length > n_ctx

Offen
#831 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
10.6k
Forks
1.4k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [X ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [ X] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [ X] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [ X] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

Embedding a document, if embedding document tokens are > to n_ctx , it should warn or exit()

# Current Behavior
it fails with error
File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in embed_documents
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 905, in embed
return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 869, in create_embedding
self.eval(tokens)
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 549, in eval
self.input_ids[self.n_tokens : self.n_tokens + n_tokens] = batch
ValueError: could not broadcast input array from shape (8,) into shape (0,)

# Environment and Context
# Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

# Steps to Reproduce

document = open("DOC/mybigdocument.txt,"r")
text = document.read()
embeddings = LlamaCppEmbeddings(model_path=model_path,verbose=False)
embeddings.embed_documents([text])
# Failure Logs

Pl File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in embed_documents
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 113, in
embeddings = [self.client.embed(text) for text in texts]
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 905, in embed
return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 869, in create_embedding
self.eval(tokens)
File "/home/mac/chatest/lib/python3.10/site-packages/llama_cpp/llama.py", line 549, in eval
self.input_ids[self.n_tokens : self.n_tokens + n_tokens] = batch
ValueError: could not broadcast input array from shape (8,) into shape (0,)

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.