Broken GPU inference kernel output on Turing

Dear All,

I tested the code on a GeForce 2060 GPU and seems that the W2A8 matmul output is wrong.

The same code works properly with an RTX A5000 GPU (Ampere).

Has anyone else experienced this issue on Turing?

Thanks

Reference:

$ python3 test.py 
custom == np False
Shape(2560, 2560), W2A8: 10.90us, torch BF16: 39.58us
custom == np False
Shape(3840, 2560), W2A8: 11.37us, torch BF16: 53.93us
custom == np False
Shape(13824, 2560), W2A8: 10.97us, torch BF16: 173.78us
custom == np False
Shape(2560, 6912), W2A8: 10.91us, torch BF16: 92.06us
custom == np False
Shape(3200, 3200), W2A8: 11.00us, torch BF16: 52.27us
custom == np False
Shape(4800, 3200), W2A8: 11.03us, torch BF16: 75.66us
custom == np False
Shape(3200, 10240), W2A8: 10.82us, torch BF16: 157.68us
custom == np False
Shape(20480, 3200), W2A8: 10.98us, torch BF16: 304.58us

The only entry point named is test.py; start by running it on a Turing GeForce 2060 and comparing the W2A8 custom output with NumPy and the working Ampere result. Done means the W2A8 output is correct on Turing for the reported matrix shapes, with the discrepancy understood and covered by a reproducible check.