JuliaGPU / JuliaGPU/GemmKernels.jl
Using GemmKernels.jl in CUDA.jl
- Dominant language
- Julia
- Stars
- 86
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
Now that we have an FPU operator that ought to work on every GPU, we should start thinking about what needs to happen to use GemmKernels.jl in CUDA.jl for when CUBLAS isn't available. There's a couple of minor issues that we need to figure out first, so let's keep track of those here:
- [x] Support for small inputs: https://github.com/JuliaGPU/GemmKernels.jl/issues/52
- [x] Support for arbitrary input sizes: the BLAS wrapper needs to select appropriate tile sizes (AFAIU each level's shape need to be divisible by the lower-level), but it may be better to keep pow2 shapes internally and just mask out global memory reads (this is what CUTLAS does)
- [x] Support for arbitrary input types: The FPUOperator currently does not like, e.g., Float16xFloat32=Float32
- [ ] Support for arbitrary input objects, e.g., a Diagonal or ReshapedArray (without having to specialize the implementation)
- [x] Automatic selection of the best operator and kernel: WMMA when possible, FPU otherwise
- [ ] Improve FPUOperator Float16 accuracy: It is really bad compared to CUBLAS
- [ ] (optionally) Some basic (hard-coded) tuning
- [ ] (optionally) Improved benchmarks that can be run on every commit
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.