NVIDIA / NVIDIA/NeMo-Speech.cpp

TTS TN verbalizes comma in dates as "sil" (e.g. August 15, 1947)

Open
#18 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
117
Forks
29
Avg merge
4d 15h
Merged PRs (30d)
6

Description

Summary

With Sparrowhawk text normalization enabled, Magpie TTS mispronounces written dates of the form Month DD, YYYY. The comma between the day and year is verbalized as the token sil instead of being treated as punctuation / dropped.

Example input:

August 15, 1947

Observed spoken form includes an extra sil (or an unnatural pause/token) around the comma, rather than something like "August fifteenth nineteen forty seven".

This is not a missing-TN-grammars issue. TN is loaded (tn=enabled) and digits/dates otherwise normalize. The failure is specific to this comma in this date format.

Repro

  1. Serve Magpie TTS with Sparrowhawk TN (--tts.tn-model-dir pointing at the tn_configs from v0.1.0 tn_configs.tar.bz2).
  2. Synthesize: August 15, 1947
  3. Listen to the output (repeatable every session).

Also reproduced on the Magpie TTS Multilingual NIM: https://build.nvidia.com/nvidia/magpie-tts-multilingual

Expected

Comma in August 15, 1947 is punctuation. TN should drop it or map it to a pause that Magpie does not speak as a word.

Actual

A standalone comma token is rewritten to sil, which reaches Magpie as spoken text.

Root cause

Sparrowhawk maps a standalone comma to "sil" in normalizer_utils.cc:

if (word_names[i] == ",")
      word = AddWord(utt, token, "sil");

This is present in the upstream Sparrowhawk repo (not a NeMo-Speech.cpp-only fork change). Magpie then synthesizes sil as if it were a word.

Other commas (not this date pattern) do not always hit the same path, which is why it is easy to miss in QA.

Environment

  • nvcr.io/nvidia/nemo-speech.cpp:0.1.0
  • Magpie TTS multilingual GGUF + NanoCodec
  • TN grammars from the v0.1.0 GitHub release
  • Same class of failure on Magpie TTS Multilingual NIM

Request

Do not emit the verbalized sil token for a date comma (drop the comma, or keep an internal pause that does not reach the TTS tokenizer). Tracking this here so GGML and NIM stay aligned.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Sparrowhawk's normalizer_utils.cc and the standalone-comma handling described in the issue, then reproduce the behavior using the Magpie TTS setup and v0.1.0 TN grammars. Determine whether the change belongs upstream or in this integration; done means August 15, 1947 no longer sends a spoken "sil" token while dates still normalize.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
audio-video-rtc
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
60/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.