huggingface / huggingface/optimum-nvidia
Use FP8 by default when on a supported device
Open
- Dominant language
- Python
- Stars
- 1k
- Forks
- 103
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
If the user does not specify the use_fp8 flag as true or false, the default behavior should be FP8 when running on a supported device and FP16 otherwise. If the user does specify, use their option and error if running on an unsupported device
Contributor guide
Assessment
This issue has not been assessed yet.