EnzymeAD / EnzymeAD/Enzyme.jl

CUDA gemm correctness issue

Open
#2,886 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Julia
Stars
586
Forks
108
Avg merge
1d 5h
Merged PRs (30d)
44

Description

```julia
julia> using CUDA, cuDNN, Enzyme

julia> using MLDataDevices: gpu_device

julia> struct Linear{W,B}
weight::W
bias::B
end

julia> Linear(in::Integer, out::Integer) = Linear(randn(Float32, out, in), randn(Float32, out));

julia> (m::Linear)(x) = m.weight * x .+ m.bias

julia> gpu = gpu_device(force=true)

julia> model = Linear(2, 2) |> gpu;

julia> dmodel = Duplicated(model, Enzyme.make_zero(model));

julia> x = rand(Float32, 2) |> gpu
2-element CuArray{Float32, 1, CUDA.DeviceMemory}:
0.14851236
0.9353048

julia> f(m, x) = sum(m(x))
f (generic function with 1 method)

julia> ad = Enzyme.set_runtime_activity(Reverse);

julia> Enzyme.autodiff(ad, Const(f), Active, dmodel, Const(x));

julia> g = dmodel.dval;

julia> g.weight
2×2 CuArray{Float32, 2, CUDA.DeviceMemory}:
0.0 0.0
0.0 0.0
```

cc @CarloLucibello

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.