huggingface / huggingface/candle
MetaVoice-1B: fix degradation compared to Python version
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
The MetaVoice-1B model has significant degradation compared to the Python version. I believe one of the main causes is using a 64x smaller decoder model (instead of multiband diffusion and deepfilternet).
Multiband diffusion is a general purpose diffusion based model that can decode Encodec tokens (which is a Neural Audio codec, and can model diverse speech including music, and audio). So there are additional benefits to have this in the Candle codebase for any other LLMs in the audio/music/speech space.
DeepFilterNet is a powerful speech enhancement model, and so there are also additional benefits to having this within candle.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests. Compare the Candle MetaVoice-1B implementation with the Python version, focusing first on the decoder choice and the proposed multiband diffusion and DeepFilterNet alternatives. Done should address the reported quality degradation and demonstrate improved parity with the Python version.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100