OpenMOSS / OpenMOSS/MOSS-Transcribe-Diarize

Speaker tag [Sxx] sometimes missing on continuous/single-speaker audio, causing full segment collapse

Open
#40 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.1k
Forks
126
Avg merge
17h 42m
Merged PRs (30d)
5

Description

Summary

On some audio clips, the model returns valid transcribed text but omits the [Sxx] speaker tag entirely, even though the documented output format is [start_time][Sxx]text[end_time]. Observed on continuous single-speaker Arabic audio (monologue/lecture style, clear audio, no background noise/music). Other single-speaker (English) and multi-speaker (Arabic) files consistently return correct [Sxx] tags.

Reproduction

vLLM, fp8 quantized (vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code):

curl -X POST http://localhost:8055/v1/audio/transcriptions
-F model="OpenMOSS-Team/MOSS-Transcribe-Diarize"
-F file=@"sample_52s_arabic.wav"
-F response_format="json"
-F temperature="0"
-F max_completion_tokens="65536"

Response (no speaker tags):
{"text":"[0.00]القارئين معك[2.00][2.00]زيضلك من يوم ولدك[3.88]...[51.28]"}

Expected:
[0.00][S01]القارئين معك[2.00]...

Ruled out

  • Not audio quality: clip is clean speech, no music/noise/reverb.
  • Not speaker count: a separate single-speaker English clip tags correctly.
  • Not language: a separate multi-speaker Arabic clip tags correctly.
  • Not prompt-related: passing the documented diarization prompt explicitly gives byte-identical output to not passing it.
  • Not a one-off: re-running gives slightly different transcribed text (temp=0, fp8) but still omits tags both times.
  • No silence gaps to key off either — each segment's end time exactly equals the next one's start time.

Question

Is this a confidence-based fallback where the diarization head skips tagging rather than risk a wrong one? Is there a way to force tagging, or a confidence signal we can check client-side?

Happy to share the audio file privately for reproduction.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the documented vLLM command, the /v1/audio/transcriptions endpoint, and sample_52s_arabic.wav, then compare the output with and without the diarization prompt. No repository files or tests are named; done would require identifying whether missing [Sxx] tags are an intended fallback and documenting or exposing a way to force tagging or inspect confidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.