microsoft / microsoft/semantic-kernel

Feature Proposal: Add FunASR as Self-Hosted STT Connector

Open
#14,067 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C#
Stars
28.6k
Forks
4.8k
Avg merge
14h 13m
Merged PRs (30d)
18

Description

Semantic Kernel enables AI orchestration across multiple models and services. Adding speech-to-text as a native skill would enable voice-enabled AI agents and applications. FunASR (17.8K+ stars, https://github.com/modelscope/FunASR) provides:

  • SenseVoice: Ultra-fast multilingual ASR (50x faster than Whisper-large)
  • Paraformer: Production-grade ASR with timestamps and punctuation
  • Fun-ASR-Nano: Lightweight streaming ASR for edge deployment
  • OpenAI-compatible API: POST /v1/audio/transcriptions — drop-in Whisper API replacement

Since FunASR exposes an OpenAI-compatible endpoint, it can serve as a self-hosted STT backend in Semantic Kernel. Developers would configure a local FunASR server URL as their audio transcription endpoint, enabling fully self-hosted voice-to-text-to-response AI pipelines without external API dependencies.

This aligns with Semantic Kernels goal of flexible AI orchestration — FunASR adds another modality (audio) that can be combined with existing text generation skills.

Would adding FunASR as an STT connector be useful for Semantic Kernel users?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the proposed FunASR POST /v1/audio/transcriptions endpoint and compare its OpenAI-compatible behavior with Semantic Kernel's existing model and service integrations. Done would require a concrete scope for a native self-hosted STT connector, including how users configure the server URL and how it fits voice-enabled pipelines.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.