OpenBMB / OpenBMB/VoxCPM

长文本声音失真严重问题

Open
#372 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
37.8k
Forks
4.3k
Avg merge
7m
Merged PRs (30d)
1

Description

差不多300字左右就开始出现严重问题,一开始是用自己的音频数据来做复刻,发现到后面有严重的声音失真问题。测试内容如下:
参考音频:-Hl_CLJa-jg-Scene-006-18762_20-30.wav
测试结果:ref_continuation_male.wav

后来试了一下提供的参考音色:项目库里的reference_speaker.wav
从输入和推理两方面都测试了一下:Continuation以及Reference + Continuation,原始推理、nano-vllm、vllm-omni都出现了声音失真问题。
text: 各位朋友大家好,非常感谢你抽出宝贵的时间来收听这段较长的语音内容。今天我想通过这段大约三百字的文本,来完整地测试语音合成系统在长文本场景下的综合表现。首先,我们关注的是语气的自然与连贯,一段好的合成语音,应该像真人朗读一样,有轻重缓急,有恰当的停顿,而不是机械生硬地把字念出来。其次,是情感的表达,无论是平静的叙述、由衷的感叹还是好奇的提问,声音都应该随着内容自然地起伏变化。此外,长文本还格外考验系统的稳定性,比如会不会出现忽快忽慢、吞字漏字,或者到后半段音质明显下降的问题。如果这段话能够从头到尾保持清晰、自然、稳定,那么就说明这套系统在实际应用中是值得信赖的,再次感谢你的聆听。
prompt_text: 喂喂喂,说你呢,要是再不把桌子上的冰激淋吃掉,我可就要没收了哦,别以为我不知道你昨天晚上又熬夜打游戏了。下次再这样,我真的要生气啦。哼!
prompt_wav & reference_wav: reference_speaker.wav

结果:ref_continuation.wav 20s左右就失真严重了

是不是voxcpm2模型存在问题,1.5好像这个问题没有这么严重

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the long-text case with the attached reference_speaker.wav and the provided text, then compare Continuation and Reference + Continuation across the original inference path, nano-vllm, and vllm-omni. The reported result is severe distortion after about 20 seconds or 300 characters; done means identifying whether VoxCPM2 is responsible and documenting or resolving the regression relative to VoxCPM1.5.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.