huggingface / huggingface/evaluate
Cache results from `Evaluator`
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
When running an evaluation with the `Evaluator` class, it would be great to cache the results (e.g. store them as an Arrow dataset) so that one doesn't have to wait to recompute everything each time.
This would also give `evaluate` a similar developer experience to `datasets` and allow users to leverage the intuitions from one lib to the other :)
Contributor guide
Assessment
This issue has not been assessed yet.