JuliaGPU / JuliaGPU/CUDA.jl

Performance: bias add

Open
#298 5 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
Julia
Stars
1.4k
Forks
281
Avg merge
1d 7h
Merged PRs (30d)
30

Description

Copying from email thread:

Here is a very common operation used in every model (bias add) that is significantly slower with CuArrays, where do we start fixing this? Should I open an issue? Write a custom kernel? If so, where? Is this one of the ops that automatically gets compiled from Julia base? If so, maybe it is just a matter of optimizing threads/blocks.
```
julia> ak = KnetArray(rand(Float32,100,100))
julia> bk = KnetArray(rand(Float32,100))
julia> ac = CuArray(rand(Float32,100,100))
julia> bc = CuArray(rand(Float32,100))

julia> @benchmark ak .+ bk
BenchmarkTools.Trial:
memory estimate: 624 bytes
allocs estimate: 17
--------------
minimum time: 14.123 μs (0.00% GC)
median time: 15.123 μs (0.00% GC)
mean time: 16.942 μs (0.00% GC)
maximum time: 16.782 ms (0.00% GC)
--------------
samples: 10000
evals/sample: 1

julia> @benchmark ac .+ bc
BenchmarkTools.Trial:
memory estimate: 2.92 KiB
allocs estimate: 77
--------------
minimum time: 43.114 μs (0.00% GC)
median time: 44.808 μs (0.00% GC)
mean time: 45.696 μs (0.00% GC)
maximum time: 578.623 μs (0.00% GC)
--------------
samples: 10000
evals/sample: 1
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.