NVIDIA / NVIDIA/TensorRT-Model-Connect

Feature: Add Fun-CosyVoice3-0.5B-2512 native TensorRT support

Open
#1,263 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
254
Forks
58
Avg merge
1d 7h
Merged PRs (30d)
235

Description

Request type

New model or checkpoint support.

Problem and use case

Add a native CosyVoice3 family for reference-voice-conditioned text-to-speech. Users should be able to prepare a reference voice and generate speech through TensorRT-Model-Connect without a PyTorch or ONNX Runtime fallback for learned inference.

Proposed outcome

  • Keep checkpoint mapping, TensorRT graphs, reference frontend, orchestration, and validation owned by families/cosyvoice3.
  • Provide native CAMPPlus and speech-tokenizer reference preparation, speech-token LLM generation, conditioning, Flow, and HiFT waveform generation.
  • Support FP32, single-GPU, batch-one offline synthesis with 24 kHz mono output and one prepared voice per bundle.
  • Package component engines for native C++ audio generation, with explicit profile limits and errors for oversized requests or generation that reaches its limit without a stop token.
  • Retain component validation and native runtime/tokenizer contracts, and distinguish synthesis smoke checks from numerical parity and audio-quality qualification.

Model and target details

  • Checkpoint: FunAudioLLM/Fun-CosyVoice3-0.5B-2512.
  • Pinned revision: 29e01c4e8d000f4bcd70751be16fa94bf3d85a18.
  • Task: text-to-speech .
  • Initial precision: FP32.
  • Initial validation target: RTX 4070 Laptop GPU (8 GB), TensorRT 11.1 / CUDA 13.
  • Initial runtime path: explicit component build, voice preparation, and packaging, followed by native C++ generate-audio. Generic trtmc build integration remains follow-up work.

Submission checks

  • I searched open and closed issues and found no duplicate request.
  • I removed secrets, private/internal evidence, personal paths, and restricted artifacts.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the existing family structure and the proposed families/cosyvoice3 ownership boundary, then trace the explicit component build, voice preparation, packaging, and native C++ generate-audio path. Done means FP32 single-GPU batch-one synthesis works at 24 kHz mono with the pinned checkpoint, component validation and runtime/tokenizer contracts are retained, and profile-limit and stop-token errors are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, huggingface, python, pytorch
Domain
ai, backend, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.