huggingface / huggingface/evaluate
[Feature request] Add evaluator for speech2text models
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Now that Whisper is integrated into `transformers` and the large number of speech2text models on the Hub, it might be interesting having a speech2text evaluator that computes common metrics like word error rate etc.
This would be especially useful for `optimum` users, who want to know if their exported models suffer any degradation from quantization etc.
I don't have bandwidth to tackle this right now, but maybe someone in the community is excited to?
cc @mht-sharma
Contributor guide
Assessment
This issue has not been assessed yet.