mudler / mudler/LocalAI

Feature: Add FunASR as a self-hosted ASR backend

Open
#10,073 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

area/backends enhancement new-backend
Dominant language
Go
Stars
49.2k
Forks
4.5k
Avg merge
1d 3m
Merged PRs (30d)
239

Description

Implementation status

PR #10090 adds a local FunASR backend to LocalAI's existing audio-transcription API.

The current backend:

  • loads iic/SenseVoiceSmall through FunASR;
  • provides explicit CPU, CUDA 11, CUDA 12, and Intel profiles;
  • includes Torch and Torchaudio in profiles that install Torch;
  • accepts LocalAI's language and transcription request options;
  • strips SenseVoice language, emotion, and audio-event markup before returning OpenAI-compatible transcript text;
  • runs through LocalAI's existing backend install, test, gRPC LoadModel, and AudioTranscription paths.

SenseVoiceSmall supports Mandarin, Cantonese, English, Japanese, and Korean. Long-audio segmentation can be composed with the separate FSMN-VAD model. Speaker diarization is not a native SenseVoice output and is not claimed by this backend.

Validation

The PR has been exercised on ind-gpu8 with:

  • six focused tests;
  • real CPU and CUDA SenseVoiceSmall inference;
  • the actual LocalAI installer and shared unit-test helper;
  • gRPC model loading and transcription;
  • shell syntax, Python compilation, and diff checks.

The PR is clean and mergeable; its remaining gate is maintainer review.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with PR #10090 and review how the FunASR backend connects to LocalAI’s existing backend installer, gRPC LoadModel path, and AudioTranscription API. Check the six focused tests and the documented CPU, CUDA, Intel, and real-inference validation; done means maintainer review and a clean, mergeable implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, grpc, python
Domain
api, backend, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.