google / google/praxis

Support for loading fp8 checkpoint

Open
#68 12 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
200
Forks
45
PR merge metrics
No merged PRs in 30d

Description

There is a use_fp flag for the offline_quantize tool in saxml/tool to quantize the weight in fp8 but still has to be stored in int8(https://github.com/google/praxis/blob/3f4cbb4bcda366db7b018695fbe2d40868d19e3e/praxis/layers/quantization/operations.py#L776). If that is always the case, is there any example showcasing how to load a checkpoint in int8 but interpret as fp8? @jianlijianli @zhangqiaorjc

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.