crisishistory / crisishistory/HistoryAIToolkit

Experiment with adding speaker diarization

Open
#45 3 comments 0 reactions 1 assignee Claimed by @Subramaniam-dot View on GitHub
good first issue hacktoberfest-accepted help wanted high priority
Dominant language
Python
Stars
13
Forks
18
PR merge metrics
No merged PRs in 30d

Description

Speaker diarization is where you annotate a transcript by noting which words were spoken by which speakers.

There are tools in Python that do this. It would be great to try them out and see if any would work for our project:
* https://github.com/pyannote/pyannote-audio
* https://github.com/espnet/espnet
* Anything else anyone can find!

It's possible we may also have to implement our own speaker diarization, either here or in a separate repo that we use as a dependency here. I attended [a talk last night](https://www.meetup.com/london-ai-tech-talk/) about how News UK did this with their own dynamic clustering of their vectorized embeddings. They used the large whisper model to transcribe their audio files, and then they implemented speaker diarization using their own algorithm. I vaguely recall they used https://github.com/NVIDIA/NeMo for the auto-clustering.

Contributions welcome from anyone who wants to play with this!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.