huggingface / huggingface/evaluate

[Feature request] Add evaluator for speech2text models

Open
#324 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

Now that Whisper is integrated into `transformers` and the large number of speech2text models on the Hub, it might be interesting having a speech2text evaluator that computes common metrics like word error rate etc.

This would be especially useful for `optimum` users, who want to know if their exported models suffer any degradation from quantization etc.

I don't have bandwidth to tackle this right now, but maybe someone in the community is excited to?

cc @mht-sharma

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.