huggingface / huggingface/candle

MetaVoice-1B: fix degradation compared to Python version

Open
#1,801 7 comments 2 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

The MetaVoice-1B model has significant degradation compared to the Python version. I believe one of the main causes is using a 64x smaller decoder model (instead of multiband diffusion and deepfilternet).

Multiband diffusion is a general purpose diffusion based model that can decode Encodec tokens (which is a Neural Audio codec, and can model diverse speech including music, and audio). So there are additional benefits to have this in the Candle codebase for any other LLMs in the audio/music/speech space.

DeepFilterNet is a powerful speech enhancement model, and so there are also additional benefits to having this within candle.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files or tests. Compare the Candle MetaVoice-1B implementation with the Python version, focusing first on the decoder choice and the proposed multiband diffusion and DeepFilterNet alternatives. Done should address the reported quality degradation and demonstrate improved parity with the Python version.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.