bugbakery / bugbakery/transcribee
First-pass quality control step for low-confidence / non-dictionary words
- Dominant language
- TypeScript
- Stars
- 515
- Forks
- 39
- Avg merge
- 19h 36m
- Merged PRs (30d)
- 15
Description
Whisper's confidence scores are pretty good and usually correlate well with the quality of the output, so it would be great to be able to quickly correct all low-confidence words at once before going into finer quality control. Similarly, non-dictionary words could be flagged for manual control as well.
To stick with my example from cccamp23, I would imagine it working a bit like this:
1. I upload the audio and wait for the worker to finish
2. I click a button to start quick correction mode
3. The transcribee editor walks me through all tokens, one by one, that have a low confidence score (threshold for dark red text), in context, and lets me edit the instance, edit all instances of the same token, or mark the token as correct (which would raise the confidence for the same token in the following text)
In the aforementioned talk, I could quickly correct different spellings of "Bonify" or "Schufa" to a single spelling and mark all instances as "high confidence" that way.
On top of using whisper's confidence scores, it could be useful to also run the transcribed text through a dictionary checker, because Whisper will sometimes transcribe things like "Geburtstartum" oder "Kreditauskunftsteil" with high confidence even though those aren't dictionary words.
Contributor guide
Research direction
The transcribee editor is the entry point described by the issue. Start by locating how low-confidence tokens and repeated token instances are presented, then define the quick correction flow around confidence scores and dictionary checks. Done means users can review flagged tokens in context, edit one or all matching instances, and mark recognized words as correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- frontend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100