bugbakery / bugbakery/transcribee

First-pass quality control step for low-confidence / non-dictionary words

Open
#351 0 comments 0 reactions 0 assignees View on GitHub
enhancement frontend
Dominant language
TypeScript
Stars
515
Forks
39
Avg merge
19h 36m
Merged PRs (30d)
15

Description

Whisper's confidence scores are pretty good and usually correlate well with the quality of the output, so it would be great to be able to quickly correct all low-confidence words at once before going into finer quality control. Similarly, non-dictionary words could be flagged for manual control as well.

To stick with my example from cccamp23, I would imagine it working a bit like this:

1. I upload the audio and wait for the worker to finish
2. I click a button to start quick correction mode
3. The transcribee editor walks me through all tokens, one by one, that have a low confidence score (threshold for dark red text), in context, and lets me edit the instance, edit all instances of the same token, or mark the token as correct (which would raise the confidence for the same token in the following text)

In the aforementioned talk, I could quickly correct different spellings of "Bonify" or "Schufa" to a single spelling and mark all instances as "high confidence" that way.

On top of using whisper's confidence scores, it could be useful to also run the transcribed text through a dictionary checker, because Whisper will sometimes transcribe things like "Geburtstartum" oder "Kreditauskunftsteil" with high confidence even though those aren't dictionary words.

Contributor guide

Open the contributing guide

Research direction

The transcribee editor is the entry point described by the issue. Start by locating how low-confidence tokens and repeated token instances are presented, then define the quick correction flow around confidence scores and dictionary checks. Done means users can review flagged tokens in context, edit one or all matching instances, and mark recognized words as correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.