huggingface / huggingface/cookbook

"RAG with unstructured data", uses `documents` instead of `docs` / unused `docs` variable?

Open
#183 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.7k
Forks
417
Avg merge
17h
Merged PRs (30d)
3

Description

https://github.com/huggingface/cookbook/blob/main/notebooks/en/rag_with_unstructured_data.ipynb

Quote"
Setting up the retriever
This example uses ChromaDB as a vector store and [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model, feel free to use any other vector store.

```
from langchain_community.vectorstores import Chroma
from langchain.embeddings import HuggingFaceEmbeddings

from langchain.vectorstores import utils as chromautils

# ChromaDB doesn't support complex metadata, e.g. lists, so we drop it here.
# If you're using a different vector store, you may not need to do this
docs = chromautils.filter_complex_metadata(documents)

embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-base-en-v1.5")
vectorstore = Chroma.from_documents(documents, embeddings)
retriever = vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": 3})
```
"

Should the `documents` parameter be replaced with `docs` on the second to last line, i.e. `vectorstore = Chroma.from_documents(docs, embeddings)`? Or is this intentional? I'm not familiar with Chroma (hence why I was using this tutorial), but I did wonder what the `docs` variable was for when going through the tutorial as it didn't seem to have been used anywhere. It seems like `docs` is a filtered version of `documents`, in which case it would be passed to `from_documents` (please correct me if that is not the case). If the `docs` variable is actually used somehow (in which case my bad), or if `documents` is meant to be replaced with `docs` as I think it might, it wasn't clear.

Contributor guide

No contributing guide indexed for this repository

Research direction

Open notebooks/en/rag_with_unstructured_data.ipynb and inspect the retriever setup cell, especially the filtered docs assignment and the subsequent Chroma.from_documents call. Confirm whether the filtered variable is intended as the input, then update the notebook or clarify the intentional use and run the relevant notebook cells to verify the retriever still works.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
databases, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.