JuliaGPU / JuliaGPU/CUDA.jl

Use GemmKernels.jl for fallback mul!

Open
#1,990 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement performance
Dominant language
Julia
Stars
1.4k
Forks
281
Avg merge
1d 7h
Merged PRs (30d)
30

Description

GemmKernels.jl is shaping up to be usable for replacing the GPUArrays fallback matmul implementation, which much better performance. For example a 2048x2048x2048 Float32 matmul.

GPUArrays:

julia> @benchmark CUDA.@sync GPUArrays.generic_matmatmul!(dC, dA, dB, true, false)
BenchmarkTools.Trial: 576 samples with 1 evaluation.
 Range (min … max):  6.001 ms …   9.519 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     8.698 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   8.677 ms ± 352.550 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                                 ▃ █▅▁
  ▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▂▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▂▂▃▃▃▄▅▅█▇████▅▅▄▃▃▃ ▃
  6 ms            Histogram: frequency by time        9.22 ms <

 Memory estimate: 4.81 KiB, allocs estimate: 88.

GemmKernels:

julia> @benchmark CUDA.@sync GemmKernels.mul!(dC, dA, dB)
BenchmarkTools.Trial: 7421 samples with 1 evaluation.
 Range (min … max):  579.075 μs …  7.580 ms  ┊ GC (min … max): 0.00% … 81.60%
 Time  (median):     670.235 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   669.440 μs ± 81.284 μs  ┊ GC (mean ± σ):  0.12% ±  0.95%

                                              ▁  ▂█▅▂
  ▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▂▁▁▂▂▁▁▁▁▁▁▁▁▂▂▂▂▂▂▂▁▂▂▃▂▃▆▆█▄▅████▆▄▃▃▃▂▂▂▂ ▃
  579 μs          Histogram: frequency by time          692 μs <

 Memory estimate: 6.34 KiB, allocs estimate: 115.

CUBLAS:

julia> @benchmark CUDA.@sync mul!(dC, dA, dB)
BenchmarkTools.Trial: 9659 samples with 1 evaluation.
 Range (min … max):  404.287 μs … 593.765 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     513.306 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   512.696 μs ±  10.767 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                            ▁    ▁ ▁▆▇██▇▅▃▃▂▁  ▂
  ▄██▆▃▁▁▁▁▃▁▁▁▁▁▃▅▁▁▁▁▁▃▃▁▁▁▅█▄▁▃▁▁▃▁▅█▅▄▄▆███████████████████ █
  404 μs        Histogram: log(frequency) by time        530 μs <

 Memory estimate: 3.81 KiB, allocs estimate: 84.

And for completion, OpenBLAS:

julia> @benchmark mul!(C, A, B)
BenchmarkTools.Trial: 376 samples with 1 evaluation.
 Range (min … max):  12.956 ms …  15.188 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     13.197 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   13.294 ms ± 356.341 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▁▅▄▇███▇▅▅▃▂▅▂▂▁
  ████████████████▆▁▆▁▅▆▆▅▅▅▁▇▁█▅▅▁▅▅▁▁▅▁▁▁▁▁▁▅▁▅▅▅▁▁▁▁▁▅▅▁▁▁▅ ▇
  13 ms         Histogram: log(frequency) by time      15.1 ms <

 Memory estimate: 0 bytes, allocs estimate: 0.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the GPUArrays fallback matmul implementation and comparing its entry point with GemmKernels.mul!, using the benchmark examples in this issue as the baseline. Determine the integration scope and verify that fallback multiplication preserves expected behavior while improving performance toward the reported GemmKernels results.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
hpc
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.