长文本声音失真严重问题
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 37.8k
- Forks
- 4.3k
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
差不多300字左右就开始出现严重问题,一开始是用自己的音频数据来做复刻,发现到后面有严重的声音失真问题。测试内容如下:
参考音频:-Hl_CLJa-jg-Scene-006-18762_20-30.wav
测试结果:ref_continuation_male.wav
后来试了一下提供的参考音色:项目库里的reference_speaker.wav
从输入和推理两方面都测试了一下:Continuation以及Reference + Continuation,原始推理、nano-vllm、vllm-omni都出现了声音失真问题。
text: 各位朋友大家好,非常感谢你抽出宝贵的时间来收听这段较长的语音内容。今天我想通过这段大约三百字的文本,来完整地测试语音合成系统在长文本场景下的综合表现。首先,我们关注的是语气的自然与连贯,一段好的合成语音,应该像真人朗读一样,有轻重缓急,有恰当的停顿,而不是机械生硬地把字念出来。其次,是情感的表达,无论是平静的叙述、由衷的感叹还是好奇的提问,声音都应该随着内容自然地起伏变化。此外,长文本还格外考验系统的稳定性,比如会不会出现忽快忽慢、吞字漏字,或者到后半段音质明显下降的问题。如果这段话能够从头到尾保持清晰、自然、稳定,那么就说明这套系统在实际应用中是值得信赖的,再次感谢你的聆听。
prompt_text: 喂喂喂,说你呢,要是再不把桌子上的冰激淋吃掉,我可就要没收了哦,别以为我不知道你昨天晚上又熬夜打游戏了。下次再这样,我真的要生气啦。哼!
prompt_wav & reference_wav: reference_speaker.wav
结果:ref_continuation.wav 20s左右就失真严重了
是不是voxcpm2模型存在问题,1.5好像这个问题没有这么严重
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the long-text case with the attached reference_speaker.wav and the provided text, then compare Continuation and Reference + Continuation across the original inference path, nano-vllm, and vllm-omni. The reported result is severe distortion after about 20 seconds or 300 characters; done means identifying whether VoxCPM2 is responsible and documenting or resolving the regression relative to VoxCPM1.5.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100