microsoft / microsoft/onnxruntime
[Bug] Wrong Conv behavior using CUDAExecutionProvider in C# when kernel_shape=2
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
I previously worked with Microsoft.ML.Onnxruntime.GPU 1.16.0 CUDAExecutionProvider, and everything ran fine when using a Conv layer with kernel_shape=2.
However, after upgrading to 1.22.0, the output of my segmentation model became incorrect. The model still executes without errors, but the output image shows a completely wrong pattern.
If I change the Conv layer to use kernel_shape=3 and retrain the model, the results turn out correct again. Also, If I keep kernel_shape=2 but switch to CPUExecutionProvider, the results turn out correct. This makes me suspect there’s an issue with kernel_shape=2 in the newer version of CUDAExecutionProvider.
In my use case, I need to maintain backward compatibility for existing models, so simply switching from kernel_shape=2 to 3 is not an option.
### To reproduce
build a segmentation model which contains Conv:
dialations: 1,1
kernel_shape: 2,2
pads: 0,0,1,1
strides: 1,1
### Urgency
_No response_
### Platform
Windows
### OS Version
22H2
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.22.0
### ONNX Runtime API
C#
### Architecture
X64
### Execution Provider
CUDA
### Execution Provider Library Version
12.8
Contributor guide
Assessment
This issue has not been assessed yet.