deepset-ai / deepset-ai/haystack-core-integrations
Add ability to use custom metrics in the DeepEval Integration
@sjrl is already working on this.
Since Aug 24, 2026.
- Dominant language
- Python
- Stars
- 203
- Forks
- 332
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 80
Description
Is your feature request related to a problem? Please describe.
The DeepEval Haystack integration currently limits users to the metrics explicitly supported by the integration, preventing use of custom DeepEval metrics.
Describe the solution you'd like
Allow users to pass custom DeepEval metrics to the evaluator component, alongside the existing built-in metrics.
Describe alternatives you've considered
Running custom DeepEval metrics outside the Haystack evaluation pipeline.
Additional context
DeepEval supports implementing custom metrics, so exposing this capability would make the integration more flexible and extensible.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.