MiniMax-AI / MiniMax-AI/MiniMax-MCP

[BadCase about the model]: CN-NewsTTS Bench includes MiniMax TTS results - metadata confirmation invited

Open
#93 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
284
PR merge metrics
No merged PRs in 30d

Description

Basic Information - Models Used

minimax_tts-speech-2.8-hd

Basic Information - Scenario Description

TTS systems still struggle with Chinese news-style text

Is this badcase known and solvable?
Information about environment

git clone https://github.com/Jayden-X-L/cn-news-tts-bench.git
cd cn-news-tts-bench

python3 scripts/validate_dataset.py data/dev.jsonl
python3 scripts/validate_dataset.py data/test_public.jsonl

python3 scripts/score_submission.py
--dataset data/test_public.jsonl
--asr-results results/asr_results/public_test/volcengine_tts.asr.jsonl
--model-id volcengine_tts
--output-dir /tmp/cn-news-tts-repro

python3 scripts/aggregate_leaderboard.py
--per-model-dir results/per_model_public_test
--results-dir /tmp/cn-news-tts-leaderboard/results
--site-dir /tmp/cn-news-tts-leaderboard/site

shasum -a 256 -c release/v0.1_core_checksums.sha256

Call & Execution Information
  • model_id: minimax_tts
  • model name: speech-2.8-hd
  • voice: configured Mandarin news voice
Description

Hi MiniMax team,

I am Shijun Luo from NetEase Cloud Music, where I work on AI news briefing / AI news podcast generation. In our workflow, we use and evaluate major TTS systems to generate spoken news and information podcast content.

During this work, we found that many current TTS systems still struggle with Chinese news-style text, especially compact expressions that frequently appear in real news. These errors are not just voice-quality issues; they can change the information heard by listeners.

For example:

  • 苏-27 may be read as "苏负二十七" instead of the intended aircraft model name.
  • 96-91 may be read as a numeric range instead of a sports score.
  • 620N·m may be read letter by letter or as symbol fragments instead of a torque unit.
  • 3.5% may be read as "三点五百分号" or confused with percentage points.
  • AI / CEO may be expanded into "人工智能" / "首席执行官" when the original abbreviation should be preserved.

This motivated us to release CN-NewsTTS Bench, a raw-input Chinese news TTS benchmark focused on real-world news reading cases such as dates, numbers, units, named entities, mixed-script text, and text normalization.

The current public leaderboard includes a MiniMax TTS entry:

  • model_id: minimax_tts
  • model name: speech-2.8-hd
  • voice: configured Mandarin news voice

Repository:
https://github.com/Jayden-X-L/cn-news-tts-bench

We would like to invite the MiniMax team to:

  1. Confirm or correct the public model metadata.
  2. Submit an official result if the current configuration is not representative.
  3. Provide a system/model card if available.

Submission guide:
https://github.com/Jayden-X-L/cn-news-tts-bench/blob/main/SUBMIT.md

For questions or corrections, feel free to contact me:
xiaobiluo@gmail.com

Thanks!

Best,
Shijun Luo

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading SUBMIT.md and the validation commands for data/dev.jsonl and data/test_public.jsonl. Review the public leaderboard entry for minimax_tts and determine whether the model metadata can be confirmed or corrected; done requires an official confirmation, corrected metadata, or an official result/model card.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.