CUDA gemm correctness issue
- Dominant language
- Julia
- Stars
- 586
- Forks
- 108
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 44
Description
```julia
julia> using CUDA, cuDNN, Enzyme
julia> using MLDataDevices: gpu_device
julia> struct Linear{W,B}
weight::W
bias::B
end
julia> Linear(in::Integer, out::Integer) = Linear(randn(Float32, out, in), randn(Float32, out));
julia> (m::Linear)(x) = m.weight * x .+ m.bias
julia> gpu = gpu_device(force=true)
julia> model = Linear(2, 2) |> gpu;
julia> dmodel = Duplicated(model, Enzyme.make_zero(model));
julia> x = rand(Float32, 2) |> gpu
2-element CuArray{Float32, 1, CUDA.DeviceMemory}:
0.14851236
0.9353048
julia> f(m, x) = sum(m(x))
f (generic function with 1 method)
julia> ad = Enzyme.set_runtime_activity(Reverse);
julia> Enzyme.autodiff(ad, Const(f), Active, dmodel, Const(x));
julia> g = dmodel.dval;
julia> g.weight
2×2 CuArray{Float32, 2, CUDA.DeviceMemory}:
0.0 0.0
0.0 0.0
```
cc @CarloLucibello
Contributor guide
Assessment
This issue has not been assessed yet.