NVIDIA / NVIDIA/TensorRT-Model-Connect
Feature: Add Fun-CosyVoice3-0.5B-2512 native TensorRT support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 254
- Forks
- 58
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 235
Description
Request type
New model or checkpoint support.
Problem and use case
Add a native CosyVoice3 family for reference-voice-conditioned text-to-speech. Users should be able to prepare a reference voice and generate speech through TensorRT-Model-Connect without a PyTorch or ONNX Runtime fallback for learned inference.
Proposed outcome
- Keep checkpoint mapping, TensorRT graphs, reference frontend, orchestration, and validation owned by
families/cosyvoice3. - Provide native CAMPPlus and speech-tokenizer reference preparation, speech-token LLM generation, conditioning, Flow, and HiFT waveform generation.
- Support FP32, single-GPU, batch-one offline synthesis with 24 kHz mono output and one prepared voice per bundle.
- Package component engines for native C++ audio generation, with explicit profile limits and errors for oversized requests or generation that reaches its limit without a stop token.
- Retain component validation and native runtime/tokenizer contracts, and distinguish synthesis smoke checks from numerical parity and audio-quality qualification.
Model and target details
- Checkpoint: FunAudioLLM/Fun-CosyVoice3-0.5B-2512.
- Pinned revision:
29e01c4e8d000f4bcd70751be16fa94bf3d85a18. - Task:
text-to-speech. - Initial precision: FP32.
- Initial validation target: RTX 4070 Laptop GPU (8 GB), TensorRT 11.1 / CUDA 13.
- Initial runtime path: explicit component build, voice preparation, and packaging, followed by native C++
generate-audio. Generictrtmc buildintegration remains follow-up work.
Submission checks
- I searched open and closed issues and found no duplicate request.
- I removed secrets, private/internal evidence, personal paths, and restricted artifacts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the existing family structure and the proposed families/cosyvoice3 ownership boundary, then trace the explicit component build, voice preparation, packaging, and native C++ generate-audio path. Done means FP32 single-GPU batch-one synthesis works at 24 kHz mono with the pinned checkpoint, component validation and runtime/tokenizer contracts are retained, and profile-limit and stop-token errors are covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, huggingface, python, pytorch
- Domain
- ai, backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100