microsoft / microsoft/onnxruntime
Pad behavior is inconsistent with ONNX ReferenceEvaluator and documentation
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
# behavior
In mode 'reflect', the result from onnxscript and onnxruntime is inconsistent with onnx.reference.ReferenceEvaluator, [onnx documentation](https://onnx.ai/onnx/operators/onnx__Pad.html), and numpy.pad.
In the eager mode of onnxscript (`res_onnxscript2`), the computation is indeed conduct by onnxruntime, right? But is the `res_onnxscript1` from onnxscript evaluator? I don't know much about onnxscript internal. But the result from onnxruntime is defenitely inconsistent, so I also created an issue there.
# code to reproduce
```python
import numpy as np
from onnxscript import script, FLOAT, opset20 as op
import onnx
data = np.array([
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
], dtype = np.float32)
mode = 'reflect'
res_np = np.pad(data, ((0, 0), (2, 0)), mode = mode)
res_onnxscript1 = op.Pad(data, pads = [0, 2, 0, 0], mode = mode)
@script()
def test(data: FLOAT[None, None]) -> FLOAT[None, None]:
return op.Pad(data, pads = [0, 2, 0, 0], mode = mode)
res_onnxscript2 = test(data)
model = test.to_model_proto()
feeds = {'data': data,}
from onnx.reference import ReferenceEvaluator
sess = ReferenceEvaluator(model)
outpus_onnx = sess.run(None, feeds)
from onnxruntime import InferenceSession
sess = InferenceSession(model.SerializeToString())
outpus_onnxruntime = sess.run(None, feeds)
print(f'numpy:\n{res_np}\n')
print(f'onnxscript evaluator:\n{res_onnxscript1}\n')
print(f'onnxscript evaluator (called onnxruntime?):\n{res_onnxscript2}\n')
print(f'onnx ReferenceEvaluator:\n{outpus_onnx[0]}\n')
print(f'onnxruntime:\n{outpus_onnxruntime[0]}\n')
print(onnx.printer.to_text(model))
```
# result
```
numpy:
[[1. 1.2 1. 1.2]
[2.3 3.4 2.3 3.4]
[4.5 5.7 4.5 5.7]]
onnxscript evaluator:
Tensor(array([[0. , 1.2, 1. , 1.2],
[0. , 3.4, 2.3, 3.4],
[0. , 5.7, 4.5, 5.7]], dtype=float32))
onnxscript evaluator (called onnxruntime?):
[[0. 1.2 1. 1.2]
[0. 3.4 2.3 3.4]
[0. 5.7 4.5 5.7]]
onnx ReferenceEvaluator:
[[1. 1.2 1. 1.2]
[2.3 3.4 2.3 3.4]
[4.5 5.7 4.5 5.7]]
onnxruntime:
[[0. 1.2 1. 1.2]
[0. 3.4 2.3 3.4]
[0. 5.7 4.5 5.7]]
<
ir_version: 9,
opset_import: ["" : 20]
>
test (float[?,?] data) => (float[?,?] return_val) {
const = Constant ()
return_val = Pad (data, const)
}
```
Contributor guide
Assessment
This issue has not been assessed yet.