CouncilDataProject / CouncilDataProject/cdp-backend
Speaker classification
- Dominant language
- Python
- Stars
- 25
- Forks
- 28
- PR merge metrics
- No merged PRs in 30d
Description
### Feature Description
Backend issue for the relevant [roadmap issue
](https://github.com/CouncilDataProject/cdp-roadmap/issues/5)
Adding speaker classification to CDP transcripts. This could be through a script/class that retroactively attaches the speaker name to a transcript that already has [speaker diarization enabled](https://cloud.google.com/speech-to-text/docs/multiple-voices). Prodigy can be used for annotating the training data.
### Use Case
With speaker classification we can provide transcripts annotated with the speaker. This can be used in many ways such as through a script or github action
### Solution
Very high level idea would be to:
- Use GCP's built-in speaker diarization to separate the speakers. We could also create our own audio classification model. We could also use something like Prodigy to annotate the data, but I believe they have their own diarization/transcription models as well.
- Figure out how to add the classified speaker names to the diarized transcript. I'm not sure if GCP allows you to provide any training data, but from what I could tell they only separate the speakers, but the models don't take in training data to label them.
A bigger picture breakdown of all the major components can be found on the roadmap issue under "Major components".
Contributor guide
Assessment
This issue has not been assessed yet.