zilliztech / zilliztech/vector-graph-rag
Support incremental knowledge base updates
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 245
- Forks
- 38
- PR merge metrics
- No merged PRs in 30d
Description
Currently the add_documents method drops all collections before recreating them and importing the documents. This requires all documents to be provided to the add_documents method whenever calling it which makes the update latency grow with the amount of documents.
Instead, it would be great if there would be a way to update the knowledge base incrementally.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/vector_graph_rag/rag.py at the add_documents method and inspect the collection recreation described around line 372. Trace how repeated calls currently rebuild the knowledge base. Done means later calls can add documents incrementally without requiring all previous documents or dropping the existing collections.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100