TEN-framework / TEN-framework/ten-framework

Feature Request: Add FunASR STT extension

Open
#2,174 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
11.1k
Forks
1.4k
Avg merge
2d 15m
Merged PRs (30d)
22

Description

[!NOTE]
License and capability clarification (2026-07-14): FunASR is a toolkit, not a single checkpoint. The FunASR and SenseVoice repository source code is MIT; model weights follow each model card. SenseVoiceSmall supports Chinese, Cantonese, English, Japanese, and Korean, and its weights use the linked FunASR Model Open Source License Agreement. Fun-ASR-Nano-2512 is Apache-2.0. Language coverage, punctuation, and performance depend on the selected model and runtime configuration.

Hi! TEN framework is an excellent platform for building conversational voice AI agents.

I noticed there's been community interest in FunASR integration (#1509). I'd like to formally suggest adding a funasr_stt_python extension alongside the existing whisper_stt_python:

Why FunASR for TEN:

  • 170x real-time GPU speed: Critical for low-latency voice agents
  • Native streaming ASR: Paraformer-streaming designed for real-time with sub-second latency
  • Built-in VAD + punctuation: Simplifies the audio pipeline
  • 50+ languages: SenseVoice model with automatic language detection
  • Speaker diarization + emotion detection: Rich metadata for agent responses
  • OpenAI-compatible API: funasr-server --device cuda

FunASR is already widely used in voice agent frameworks (Fay 12.8K stars, Pipecat 12.5K stars, LiveKit 10.7K stars).

Quick integration:

from funasr import AutoModel
model = AutoModel(model="iic/SenseVoiceSmall")
result = model.generate(input=audio_bytes)

Happy to contribute a PR with the extension implementation!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the existing whisper_stt_python extension and its integration points, then compare them with FunASR's AutoModel usage and the selected model's license and language capabilities. Done means a documented FunASR STT extension is integrated with the framework and its streaming, model, and metadata behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, audio-video-rtc
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.