JuliaGPU / JuliaGPU/GemmKernels.jl

Using GemmKernels.jl in CUDA.jl

Open
#108 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Julia
Stars
86
Forks
13
PR merge metrics
No merged PRs in 30d

Description

Now that we have an FPU operator that ought to work on every GPU, we should start thinking about what needs to happen to use GemmKernels.jl in CUDA.jl for when CUBLAS isn't available. There's a couple of minor issues that we need to figure out first, so let's keep track of those here:

- [x] Support for small inputs: https://github.com/JuliaGPU/GemmKernels.jl/issues/52
- [x] Support for arbitrary input sizes: the BLAS wrapper needs to select appropriate tile sizes (AFAIU each level's shape need to be divisible by the lower-level), but it may be better to keep pow2 shapes internally and just mask out global memory reads (this is what CUTLAS does)
- [x] Support for arbitrary input types: The FPUOperator currently does not like, e.g., Float16xFloat32=Float32
- [ ] Support for arbitrary input objects, e.g., a Diagonal or ReshapedArray (without having to specialize the implementation)
- [x] Automatic selection of the best operator and kernel: WMMA when possible, FPU otherwise
- [ ] Improve FPUOperator Float16 accuracy: It is really bad compared to CUBLAS
- [ ] (optionally) Some basic (hard-coded) tuning
- [ ] (optionally) Improved benchmarks that can be run on every commit

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.