deepset-ai / deepset-ai/haystack-core-integrations
ragas: Add support for more providers in serialization/deserialization for the ragas evaluator
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 203
- Forks
- 332
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 80
Description
In PR https://github.com/deepset-ai/haystack-core-integrations/pull/3207 we added se/de support for the RAGAS evaluator component but it currently only supports deserialization when OpenAI models are used for the metric evaluation.
We could extend the support for other providers like google but the RAGAS library is no longer actively maintained so this is not a high priority.
👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing PR #3207 to understand the existing RAGAS evaluator serialization and deserialization support. Trace how OpenAI models are handled and determine where another provider, such as Google, would fit. Done means non-OpenAI provider configurations can also be deserialized for metric evaluation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100