Abraxas-365 / Abraxas-365/langchain-rust
Ability to add custom ids for documents
- Ngôn ngữ chính
- Rust
- Star
- 1.3k
- Fork
- 177
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
I want to add custom IDs for documents so that when I do re-indexing for updates, I can reference them with the ID I have in the data source.
My specific case is with the OpenSearch vector store. Right now as a workaround, I'm running a local build of this crate, and I have added a function with the following signature:
```rs
async fn add_documents_with_ids(
&self,
docs: &[Document],
opt: &VecStoreOptions,
ids: &Vec,
) -> Result, Box> {
```
and then I'm zipping the ids together with the docs and vectors
```rs
for (doc, (vector, doc_id)) in zip(docs.iter(), zip(vectors.iter(), ids.iter())) {
```
and finally adding the id to the docs
```rs
let operation = json!({"index": {
"_id": doc_id,
}});
```
I wanted to ask if someone would be interested in having this feature as well, and if yes, any suggestions from maintainers on how to implement this without having to break the core trait of VectorStore somehow.
Also, Python package has one additional field for the add_docs, called ids and you can see it in this file:
https://github.com/langchain-ai/langchain/blob/29aa9d67506ac07b92d37d58c684ce3c6dc290cd/libs/community/langchain_community/vectorstores/opensearch_vector_search.py#L587
Hướng dẫn đóng góp
Hướng nghiên cứu
Look at the VectorStore trait and the OpenSearch vector store implementation to understand how documents are added. The Python reference shows an 'ids' parameter. You'll need to modify the add_documents method signature and implementation to accept optional custom IDs, ensuring backward compatibility. Check how IDs are currently generated and where the OpenSearch _id field is set.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- rust
- Lĩnh vực
- backend, databases
- Loại issue
- Tính năng
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Đình trệ
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 45/100