deepset-ai / deepset-ai/haystack
Add `FilterBuilder` to make Haystack metadata filters easier to write and read
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 26.6k
- Forks
- 3.2k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 194
Description
Is your feature request related to a problem? Please describe.
Yes. I often have to stop and look up the exact Haystack filter syntax whenever I need anything more than a very simple filter.
Single-condition filters are manageable, but once I need nested AND / OR logic, the dictionaries get noisy very quickly. I find myself spending time counting braces, remembering whether something belongs under operator or conditions, and double-checking field/operator/value structure instead of focusing on the actual retrieval logic I want.
This becomes even more frustrating when filters are built dynamically in pipelines, apps, or agent tools, where readability matters even more.
Describe the solution you'd like
I’d love a first-party FilterBuilder that helps users construct valid Haystack filters in a more readable and less error-prone way, while still producing the same plain dictionary format Haystack already expects.
For example, something conceptually like:
filters = (
FilterBuilder()
.eq("meta.type", "article")
.gte("meta.year", 2024)
.or_group(
lambda f: f.in_("meta.genre", ["economy", "politics"]).eq("meta.publisher", "nytimes")
)
.build()
)
The exact API does not have to look like this, but the main idea is:
- make filters easier to read
- make nested logic easier to build
- reduce small syntax mistakes
- keep compatibility by returning the current filter dict format under the hood
This would be especially helpful for FilterRetriever, DocumentStore.filter_documents(), and any place where filters are built programmatically.
Describe alternatives you've considered
The current alternative is to keep writing filter dictionaries by hand. That works, but it gets hard to read and easy to mess up once the filter is even slightly complex.
I’ve also considered writing small helper functions in my own code, but those end up being one-off utilities and don’t feel consistent across projects.
Another alternative is to keep copying filter patterns from the docs and modifying them each time, but that doesn’t really solve the ergonomics or readability problem.
Additional context
This request is mainly about developer experience, not changing how filtering works.
What makes this feel worthwhile is that filter syntax is already a stable, documented part of Haystack and is used across multiple places in the framework. A builder would make that same functionality easier to use, especially for newer users.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing filter syntax and the filter entry points mentioned in FilterRetriever and DocumentStore.filter_documents(). Define the builder API and its handling of nested AND/OR groups before implementing it. Done means it produces the existing plain filter dictionary format while making dynamic filters easier to construct and read.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100