JuliaGPU / JuliaGPU/KernelAbstractions.jl
Degraded performance using kwargs in kernels
- Dominant language
- Julia
- Stars
- 523
- Forks
- 88
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 25
Description
I've heard that there's performance degradation using kwargs in kernels, so here's my attempt at a MWE: (`test/kwarg_performance.jl`)
```julia
using KernelAbstractions
using CUDA
using Test
foo_kwarg(;a=1) = a+2
foo_parg(a=1) = a+2
@testset "Kwarg performance" begin
@kernel function kwarg_performance(A, @Const(B))
I = @index(Global)
@inbounds A[I] = foo_kwarg(;a=B[I])
end
@kernel function parg_performance(A, @Const(B))
I = @index(Global)
@inbounds A[I] = foo_parg(B[I])
end
N = 1024
x = ones(N, N)
y = similar(x)
@time let
event = kwarg_performance(CPU())(y, x; ndrange=length(x))
wait(event)
end
@test y == [foo_kwarg(;a=x_i) for x_i in x]
y = similar(x)
@time let
event = parg_performance(CPU())(y, x; ndrange=length(x))
wait(event)
end
@test y == [foo_parg(x_i) for x_i in x]
if has_cuda_gpu()
cx = CuArray(x)
cy = similar(cx)
synchronize()
@time let
event = kwarg_performance(CUDADevice())(cy, cx; ndrange=length(x))
wait(event)
end
cy = Array(cy)
@test cy == [foo_kwarg(;a=x_i) for x_i in x]
cy = similar(cx)
synchronize()
@time let
event = parg_performance(CUDADevice())(cy, cx; ndrange=length(x))
wait(event)
end
cy = Array(cy)
@test cy == [foo_parg(x_i) for x_i in x]
end
end
```
For me, this yields
```julia
julia> include("test\\kwarg_performance.jl")
2.509230 seconds (6.49 M allocations: 333.492 MiB, 2.89% gc time)
0.128177 seconds (324.95 k allocations: 16.672 MiB)
┌ Warning: `haskey(::TargetIterator, name::String)` is deprecated, use `Target(; name=name) !== nothing` instead.
│ caller = llvm_compat(::VersionNumber) at compatibility.jl:181
└ @ CUDA C:\Users\kawcz\.julia\packages\CUDA\42B9G\deps\compatibility.jl:181
12.484406 seconds (26.56 M allocations: 1.307 GiB, 4.27% gc time)
0.296861 seconds (743.03 k allocations: 38.406 MiB)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)
```
for the first run. Running several more times:
```julia
julia> include("test\\kwarg_performance.jl")
0.123319 seconds (339.48 k allocations: 17.460 MiB)
0.113910 seconds (297.41 k allocations: 15.286 MiB)
0.354558 seconds (757.10 k allocations: 38.929 MiB, 3.05% gc time)
0.319298 seconds (742.97 k allocations: 38.169 MiB, 3.40% gc time)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)
julia> include("test\\kwarg_performance.jl")
0.121902 seconds (339.50 k allocations: 17.461 MiB)
0.135150 seconds (297.36 k allocations: 15.278 MiB, 6.05% gc time)
0.317421 seconds (757.07 k allocations: 38.925 MiB)
0.327761 seconds (743.02 k allocations: 38.173 MiB, 4.08% gc time)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)
julia> include("test\\kwarg_performance.jl")
0.131541 seconds (339.49 k allocations: 17.464 MiB)
0.125543 seconds (297.37 k allocations: 15.277 MiB)
0.419949 seconds (757.04 k allocations: 38.930 MiB, 27.22% gc time)
0.302847 seconds (743.00 k allocations: 38.168 MiB)
Test Summary: | Pass Total
Kwarg performance | 4 4
Test.DefaultTestSet("Kwarg performance", Any[], 4, false)
```
@lcw is there something wrong with my MWE? or is the problem with the expensive first run? Or am I somehow ineffectively measuring?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running test/kwarg_performance.jl and compare the CPU and CUDA kernel timings and allocation counts for kwarg_performance and parg_performance. Check whether the difference persists after repeated runs and determine whether the issue is kwargs handling or measurement warmup; done means the cause and reproducible behavior are documented or corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100