google / google/speaker-id

DiarizationLM: Clarification on TPST usage for preserving ASR transcript while transferring semantic speaker labels

Open
#43 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
456
Forks
41
PR merge metrics
No merged PRs in 30d

Description

Hi team,

I’m working with the `diarizationlm.transcript_preserving_speaker_transfer` (TPST) function and need some guidance on the best way to preserve the original ASR transcript while transferring semantic speaker labels (e.g., Doctor/Patient) predicted by an LLM.

### What I’ve tried

**Direct TPST from LLM → ASR:**

```python
transferred_spk_label = diarizationlm.transcript_preserving_speaker_transfer(
words_llm_label, speakers_llm_label,
words_corrected, speakers_corrected
)

# Variable meaning:
# words_llm_label: LLM-generated transcript text
# speakers_llm_label: LLM-generated speaker labels (semantic, e.g., Doctor/Patient)
# words_corrected: ASR transcript (must remain unchanged)
# speakers_corrected: ASR speaker IDs (numeric)
```

**Issue:**
The output contains **ASR numeric speaker IDs**, not the semantic labels.
**Reason:**
TPST inherently outputs labels in the **target speaker space**, so semantic labels get replaced by ASR numeric IDs.

### My Use Case

* Preserve the **exact ASR transcript** (word-for-word).
* Transfer **semantic speaker labels** from the LLM output.
* Support cases where multiple ASR speaker IDs need to be mapped to only semantic roles (eg: Doctor/Patient).

### My Question

1. Is there a recommended method or best practice to use TPST in this scenario while ensuring the ASR transcript stays fully intact **and** semantic labels are preserved/propagated correctly?

Any guidance or suggestions would be greatly appreciated.

Thank you for this great algorithm and all the work behind it!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.