Support for loading fp8 checkpoint
Open
- Dominant language
- Python
- Stars
- 200
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Description
There is a use_fp flag for the offline_quantize tool in saxml/tool to quantize the weight in fp8 but still has to be stored in int8(https://github.com/google/praxis/blob/3f4cbb4bcda366db7b018695fbe2d40868d19e3e/praxis/layers/quantization/operations.py#L776). If that is always the case, is there any example showcasing how to load a checkpoint in int8 but interpret as fp8? @jianlijianli @zhangqiaorjc
Contributor guide
Assessment
This issue has not been assessed yet.