ChatGPTNextWeb / ChatGPTNextWeb/NextChat

Feature: Add FunASR/SenseVoice as a local STT engine

Open
#6,803 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
88.8k
Forks
59.1k
PR merge metrics
No merged PRs in 30d

Description

## Goal

Let NextChat voice input use a self-hosted FunASR/SenseVoice transcription service without bundling a Python or model runtime into the NextChat application.

## Proposed first milestone

Reuse the OpenAI-compatible multipart contract:

```http
POST /v1/audio/transcriptions
Content-Type: multipart/form-data
```

The initial integration should:

- keep the existing transcription behavior as the default;
- allow a configurable transcription base URL, model name, and optional bearer token;
- send the recorded audio using the existing multipart flow and consume the standard `{ "text": "..." }` response;
- keep credentials on the existing server-side/configuration boundary and avoid leaking them into logs or shareable settings;
- work with any compatible endpoint rather than adding a hard dependency on the `funasr` Python package.

This contract can point to a local FunASR deployment while remaining useful for other self-hosted transcription servers.

## Capability and license scope

- [FunASR](https://github.com/modelscope/FunASR) is a toolkit; capabilities depend on the selected model and runtime.
- [SenseVoiceSmall](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) supports Chinese, Cantonese, English, Japanese, and Korean, plus speech-event and emotion tags. Its model card links the applicable model license.
- [Fun-ASR-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) is a separate model with Apache-2.0 weights.
- The FunASR and SenseVoice repository source code is MIT. Model weights must be evaluated under each model card license.

Performance should be documented for the exact checkpoint, hardware, dtype, audio set, and concurrency used; this issue intentionally makes no universal speed or language-count claim.

## Example request

```bash
curl -X POST "${TRANSCRIPTION_BASE_URL}/v1/audio/transcriptions" \
-H "Authorization: Bearer ${TRANSCRIPTION_API_KEY}" \
-F "file=@voice.webm" \
-F "model=${TRANSCRIPTION_MODEL}"
```

The requested product change is therefore a configurable compatible endpoint, not a FunASR-only runtime embedded in NextChat.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.