[Proposal] OpenEval Import/Export Support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Overview
We have launched OpenEval (v1.0.0-rc.1), an open standard (Apache 2.0) for portable LLM evaluation datasets. Every eval framework uses incompatible formats — OpenEval provides a standard JSON format + converters + SDKs (TypeScript + Python).
What We're Proposing
Add to_openeval() and from_openeval() methods (~50-100 lines using our SDK).
Resources
- Spec: https://github.com/adhabnr-ux/openeval/blob/main/spec/SPEC.md
- npm SDK:
npm install @openeval/sdk - PyPI SDK:
pip install openeval
We would love to collaborate.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the OpenEval spec at spec/SPEC.md and inspect the Python SDK from the linked PyPI package first. Then locate CrewAI's existing evaluation or dataset import/export entry points and determine how the proposed methods should map to the standard format; done means both to_openeval() and from_openeval() are supported with compatible data conversion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100