Performance hit of using cudadevrt
- Dominant language
- Julia
- Stars
- 1.4k
- Forks
- 281
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 30
Description
Linking the cuda device runtime incurs a performance hit, for example https://github.com/JuliaGPU/CUDA.jl/issues/799:
```julia
using BenchmarkTools, Printf, Random, CUDA
const threads = 256
#simple add matrix and vector kernel
function kernel_add_mat_vec(m, x1, x2, y)
# one block per column
offset = (blockIdx().x-1) * m
@inbounds xtmp = x2[blockIdx().x]
for i = threadIdx().x : blockDim().x : m
@inbounds y[offset + i] = x1[offset + i] + xtmp
end
return
end
function add!(y, x1, x2)
m, n = size(x1)
@cuda blocks = n, 1 threads = threads kernel_add_mat_vec(m, x1, x2, y)
end
Random.seed!(1)
m, n = 3072, 1536 # 256 multiplier
x1 = cu(randn(Float32, (m, n)) .+ Float32(0.5))
x2 = cu(randn(Float32, (1, n)) .+ Float32(0.5))
y1 = similar(x1)
add!(y1, x1, x2)
print("add! ");
@btime begin add!($y1, $x1, $x2); synchronize() end
```
118 vs 107us with or without libcudadevrt. I had assumed this would have been fixed on CUDA 11.2, but maybe we need to do something to enable link-time optimization re. https://developer.nvidia.com/blog/improving-gpu-app-performance-with-cuda-11-2-device-lto/?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.