Abraxas-365 / Abraxas-365/langchain-rust

Ability to add custom ids for documents

Abierto
#177 2 comentarios 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
Rust
Estrellas
1.3k
Forks
177
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

I want to add custom IDs for documents so that when I do re-indexing for updates, I can reference them with the ID I have in the data source.

My specific case is with the OpenSearch vector store. Right now as a workaround, I'm running a local build of this crate, and I have added a function with the following signature:

```rs
async fn add_documents_with_ids(
&self,
docs: &[Document],
opt: &VecStoreOptions,
ids: &Vec,
) -> Result, Box> {
```
and then I'm zipping the ids together with the docs and vectors

```rs
for (doc, (vector, doc_id)) in zip(docs.iter(), zip(vectors.iter(), ids.iter())) {
```

and finally adding the id to the docs

```rs
let operation = json!({"index": {
"_id": doc_id,
}});
```

I wanted to ask if someone would be interested in having this feature as well, and if yes, any suggestions from maintainers on how to implement this without having to break the core trait of VectorStore somehow.

Also, Python package has one additional field for the add_docs, called ids and you can see it in this file:
https://github.com/langchain-ai/langchain/blob/29aa9d67506ac07b92d37d58c684ce3c6dc290cd/libs/community/langchain_community/vectorstores/opensearch_vector_search.py#L587

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.