microsoft / microsoft/onnxruntime
[ORT-nightly]Float8_e4m3 results are off-by-one
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
Checked with(tried both gpu and cpu):
`pip install ort-nightly-gpu==1.16.0.dev20230729001 --extra-index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/`
Running a simple model that consists of the following:
input -> QuantizeLinear(scale=1, zero_point=Float8_e4m3) -> DequantizeLinear(scale=1, zero_point=Float8_e4m3) -> output
Generates what seems to be an off-by-one results(in respect of fp8 representable values).
For example:
when input==`1.8131605`, ORT produce `1.75`, while the expected result is `1.875`
when looking at the abs distance between the input and the results:
```
abs(1.875-1.8131605) = 0.06183950000000005
abs(1.8131605-1.75) = 0.06316049999999995
```
Hence we expect the the result would be 1.875, which is the correct round-to.
Adding onnx model, and numpy arrays for repro.
### To reproduce
Will add model and inputs
### Urgency
_No response_
### Platform
Linux
### OS Version
ubuntu20.04
### ONNX Runtime Installation
Other / Unknown
### ONNX Runtime Version or Commit ID
1.16.0.dev20230729001
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU, CUDA
### Execution Provider Library Version
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.