JuliaGPU / JuliaGPU/KernelAbstractions.jl

Degraded performance using kwargs in kernels

Open
#114 3 comments 0 reactions 0 assignees View on GitHub
needs test
Dominant language
Julia
Stars
523
Forks
88
Avg merge
1d 11h
Merged PRs (30d)
25

Description

I've heard that there's performance degradation using kwargs in kernels, so here's my attempt at a MWE: (`test/kwarg_performance.jl`)

```julia
using KernelAbstractions
using CUDA
using Test

foo_kwarg(;a=1) = a+2
foo_parg(a=1) = a+2

@testset "Kwarg performance" begin

@kernel function kwarg_performance(A, @Const(B))
I = @index(Global)
@inbounds A[I] = foo_kwarg(;a=B[I])
end
@kernel function parg_performance(A, @Const(B))
I = @index(Global)
@inbounds A[I] = foo_parg(B[I])
end

N = 1024
x = ones(N, N)

y = similar(x)
@time let
event = kwarg_performance(CPU())(y, x; ndrange=length(x))
wait(event)
end

@test y == [foo_kwarg(;a=x_i) for x_i in x]

y = similar(x)
@time let
event = parg_performance(CPU())(y, x; ndrange=length(x))
wait(event)
end

@test y == [foo_parg(x_i) for x_i in x]

if has_cuda_gpu()
cx = CuArray(x)

cy = similar(cx)
synchronize()
@time let
event = kwarg_performance(CUDADevice())(cy, cx; ndrange=length(x))
wait(event)
end

cy = Array(cy)
@test cy == [foo_kwarg(;a=x_i) for x_i in x]

cy = similar(cx)
synchronize()
@time let
event = parg_performance(CUDADevice())(cy, cx; ndrange=length(x))
wait(event)
end

cy = Array(cy)
@test cy == [foo_parg(x_i) for x_i in x]
end
end
```

For me, this yields
```julia
julia> include("test\\kwarg_performance.jl")
2.509230 seconds (6.49 M allocations: 333.492 MiB, 2.89% gc time)
0.128177 seconds (324.95 k allocations: 16.672 MiB)
┌ Warning: `haskey(::TargetIterator, name::String)` is deprecated, use `Target(; name=name) !== nothing` instead.
│ caller = llvm_compat(::VersionNumber) at compatibility.jl:181
└ @ CUDA C:\Users\kawcz\.julia\packages\CUDA\42B9G\deps\compatibility.jl:181
12.484406 seconds (26.56 M allocations: 1.307 GiB, 4.27% gc time)
0.296861 seconds (743.03 k allocations: 38.406 MiB)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)
```
for the first run. Running several more times:
```julia
julia> include("test\\kwarg_performance.jl")
0.123319 seconds (339.48 k allocations: 17.460 MiB)
0.113910 seconds (297.41 k allocations: 15.286 MiB)
0.354558 seconds (757.10 k allocations: 38.929 MiB, 3.05% gc time)
0.319298 seconds (742.97 k allocations: 38.169 MiB, 3.40% gc time)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)

julia> include("test\\kwarg_performance.jl")
0.121902 seconds (339.50 k allocations: 17.461 MiB)
0.135150 seconds (297.36 k allocations: 15.278 MiB, 6.05% gc time)
0.317421 seconds (757.07 k allocations: 38.925 MiB)
0.327761 seconds (743.02 k allocations: 38.173 MiB, 4.08% gc time)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)

julia> include("test\\kwarg_performance.jl")
0.131541 seconds (339.49 k allocations: 17.464 MiB)
0.125543 seconds (297.37 k allocations: 15.277 MiB)
0.419949 seconds (757.04 k allocations: 38.930 MiB, 27.22% gc time)
0.302847 seconds (743.00 k allocations: 38.168 MiB)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)
```

@lcw is there something wrong with my MWE? or is the problem with the expensive first run? Or am I somehow ineffectively measuring?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running test/kwarg_performance.jl and compare the CPU and CUDA kernel timings and allocation counts for kwarg_performance and parg_performance. Check whether the difference persists after repeated runs and determine whether the issue is kwargs handling or measurement warmup; done means the cause and reproducible behavior are documented or corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.