speechbrain / speechbrain/speechbrain
Feature request: Add FunASR/SenseVoice model integration
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 11.8k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
Feature Request
Add FunASR models (SenseVoice, Paraformer) as available ASR models in SpeechBrain.
Why
SpeechBrain is the go-to speech processing toolkit. FunASR models would be a valuable addition:
| Model | Params | Architecture | Languages | Speed |
|---|---|---|---|---|
| SenseVoice-Small | 234M | Non-autoregressive | 50+ | 25x RT (CPU) |
| Paraformer-large | 220M | Non-autoregressive | zh/en/ja/ko/yue | 170x RT (GPU) |
| Fun-ASR-Nano | 800M | LLM-based (encoder+decoder) | 31 | GPU via vLLM |
| cam++ | 7.2M | Speaker diarization | - | - |
| FSMN-VAD | 5.2M | Voice activity detection | - | - |
Key advantages
- Non-autoregressive: Paraformer and SenseVoice avoid hallucination issues common in autoregressive models
- Industrial-grade: Deployed at scale (1M+ pip installs/month)
- Complete pipeline: VAD + ASR + punctuation + diarization in one package
- HuggingFace models: Available on HF Hub (
funasr/paraformer-large,funasr/sensevoice-small) - Transformers integration in progress (PR #46180)
References
- FunASR — 16K+ stars
- SenseVoice paper
- Paraformer paper
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the FunASR repository and the referenced Hugging Face models for SenseVoice and Paraformer, then check Transformers integration PR #46180. Identify SpeechBrain entry points and tests for adding external ASR models. Done should include a defined, tested integration scope for the requested models and any agreed pipeline components.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100