deepset-ai / deepset-ai/haystack-core-integrations
Extending MongoDBAtlasDocumentStore to support custom schema
@bogdankostic is already working on this.
Since Jun 30, 2026.
- Dominant language
- Python
- Stars
- 203
- Forks
- 332
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 80
Description
Is your feature request related to a problem? Please describe.
The current implementation of MongoDBAtlasDocumentStore only supports specific MongoDB document schema. Content is expected to be stored in the content field, and metadata must be within a meta subdocument. This schema requirement is enforced by the $project stage in the aggregation pipeline executed by _embedding_retrieval function:
{
"$vectorSearch": {
"index": self.vector_search_index,
"path": "embedding",
"queryVector": query_embedding,
"numCandidates": 100,
"limit": top_k,
"filter": filters,
}
},
{
"$project": {
"_id": 0,
"content": 1,
"dataframe": 1,
"blob": 1,
"meta": 1,
"embedding": 1,
"score": {"$meta": "vectorSearchScore"},
}
}
This tightly couples the Haystack Document representation with the database schema, which can be inconvenient. I have a vector store in MongoDB with an existing schema defined when I was using langchaig. Specifically, I have the document's content stored in a text field, and I have some metadata stored in different fields of a MongoDB document (like source storing the original document location reference). I would prefer to avoid migrating to a new schema dictated by MongoDBAtlasDocumentStore.
Describe the solution you'd like
I propose adding the ability to override the $project stage of the aggregation pipeline partially, optionally, while retaining the existing behavior as a default. For example, initializing the MongoDBAtlasDocumentStore could look like this:
MongoDBAtlasDocumentStore(
database_name="db",
collection_name="embedded_docs",
vector_search_index='index',
content_field_key='text', # maps "text" field in MongoDB to Document's content
meta_project_mapping={
{"source": "$source"} # allows to flexibly build meta from a MongoDB doc fields
}
self.content_field_key and self.meta_project_mapping would be then used in the $project aggregation pipeline stage. What do you think?
Describe alternatives you've considered
I extended MongoDBAtlasDocumentStore in my project and made the described change. While this approach works, I was wondering if it would be beneficial to include it in the library.
Additional context
I can submit a PR :)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.