fp16 support for older hardware
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 37.8k
- Forks
- 4.3k
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
Hi, I'm a big fan of this project, and I've been using it quite a lot ever since I solved the FFMPEG issue that I had posted here over the past weeks. Through my experiments, i noticed that VoxCPM is inherently set on using bfloat16 for its precision. The problem is that I have a GTX 1650, which doesn't support this precision (as its for Triton-compatible GPUs, which mine isn't). As a result of this, the model loads and outputs audio in full-precision, which can take a lot of time.
Thus, could you guys implement a check where in case a device can't use bf16 (therefore most likely an older hardware), VoxCPM automatically switches to fp16? That would make things much more convenient.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating where VoxCPM selects bfloat16 and handles device precision, then reproduce the behavior on hardware such as the GTX 1650. Done means devices that cannot use bfloat16 automatically use fp16 instead of falling back to full precision, with the resulting model loading and audio generation behavior checked.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100