huggingface / huggingface/cookbook
"RAG with unstructured data", uses `documents` instead of `docs` / unused `docs` variable?
- Dominant language
- Jupyter Notebook
- Stars
- 2.7k
- Forks
- 417
- Avg merge
- 17h
- Merged PRs (30d)
- 3
Description
https://github.com/huggingface/cookbook/blob/main/notebooks/en/rag_with_unstructured_data.ipynb
Quote"
Setting up the retriever
This example uses ChromaDB as a vector store and [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model, feel free to use any other vector store.
```
from langchain_community.vectorstores import Chroma
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import utils as chromautils
# ChromaDB doesn't support complex metadata, e.g. lists, so we drop it here.
# If you're using a different vector store, you may not need to do this
docs = chromautils.filter_complex_metadata(documents)
embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-base-en-v1.5")
vectorstore = Chroma.from_documents(documents, embeddings)
retriever = vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": 3})
```
"
Should the `documents` parameter be replaced with `docs` on the second to last line, i.e. `vectorstore = Chroma.from_documents(docs, embeddings)`? Or is this intentional? I'm not familiar with Chroma (hence why I was using this tutorial), but I did wonder what the `docs` variable was for when going through the tutorial as it didn't seem to have been used anywhere. It seems like `docs` is a filtered version of `documents`, in which case it would be passed to `from_documents` (please correct me if that is not the case). If the `docs` variable is actually used somehow (in which case my bad), or if `documents` is meant to be replaced with `docs` as I think it might, it wasn't clear.
Contributor guide
No contributing guide indexed for this repository
Research direction
Open notebooks/en/rag_with_unstructured_data.ipynb and inspect the retriever setup cell, especially the filtered docs assignment and the subsequent Chroma.from_documents call. Confirm whether the filtered variable is intended as the input, then update the notebook or clarify the intentional use and run the relevant notebook cells to verify the retriever still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- databases, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100