JuliaGPU / JuliaGPU/KernelAbstractions.jl

Enzyme autodiff produces out-of-bounds error for some kernels.

Open
#493 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
523
Forks
88
Avg merge
1d 11h
Merged PRs (30d)
25

Description

Running this code using the current versions of Enzyme, KA, and CUDA.jl:
```
using KernelAbstractions
using CUDA

using Enzyme

function advanceTimeLevels!(field; backend=CUDABackend())

nthreads = 64

kernel2d! = advance_2d_array(backend, nthreads)

kernel2d!(field, ndrange=size(field)[1])
end

@kernel function advance_2d_array(field)
j = @index(Global, Linear)
if j < 101
@inbounds field[j,1] = field[j,2]
end
@synchronize()
end

field = CUDA.CuArray(ones(100, 2))

d_field = Enzyme.make_zero(field)

autodiff(Enzyme.Reverse, advanceTimeLevels!, Duplicated(field, d_field))

@show field
@show d_field
```
produces this error (can add more of the stacktrace if needed):
```
ERROR: a BoundsError was thrown during kernel execution on thread (37, 1, 1) in block (2, 1, 1).
Out-of-bounds array access
```
Since there are 64 threads per block, the 37th entry of block 2 corresponds to global index 101 which is out-of-bounds for the array `field`. But the kernel has a conditional statement to avoid accessing the array at any entry greater than 100 (its length). If we run the function `advanceTimeLevels!` without `autodiff`, no error occurs. If we run `autodiff` with a block size that divides the array length, such as `nthreads = 100` or `nthreads = 50`, no error as well.

@wsmoses @michel2323

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the advanceTimeLevels! and advance_2d_array reproducer, comparing direct kernel execution with Enzyme.autodiff using a block size that does not divide the array length. Trace the generated kernel's handling of the conditional and bounds access; done when the reproducer completes without an out-of-bounds error for such block sizes.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.