AccelerateHS / AccelerateHS/accelerate

GSoc Project Idea: Use Nvidia's new WMMA Execution Resources

Open
#434 6 comments 0 reactions 0 assignees View on GitHub
new feature
Dominant language
Haskell
Stars
1k
Forks
135
PR merge metrics
No merged PRs in 30d

Description

It's that time of year again, and having done some work with Nvidia's new hardware sporting "TensorRT" cores, I thought it'd make a nice project to evaluate whether or not Accelerate programs would benefit from these new execution resources and, if they do, start emitting code with accelerate-llvm-ptx that takes advantage of them.

The shiny new "TensorRT" cores are nothing more than dedicated execution resources for doing fused-multiply-accumulate on arrays of a fixed size and type. Nvidia calls their intrinsics for these operations WMMA or warp-level matrix multiply accumulate. LLVM gained support for them [here](https://reviews.llvm.org/D38645) and Nvidia's documentation for the ISA is [here](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#warp-level-matrix-instructions). I believe CUDA version 10 is required to launch kernels that use these intrinsics.

On paper(or rather, in my head) this should be fairly straightforward to implement. The PTX CodeGen needs to look for matrix operations with types and array shapes that are compatible with WMMA execution resources available on the targeted compute capability. If compatible expressions are found, WMMA intrinsics may be emitted for them. It's not obvious (at least to me) that typical (i.e. not-contrived-for-WMMA) accelerate programs are well positioned to take advantage of these intrinsics, so benchmarking is in order as well.

Although I've had the privilege to use both accelerate and LLVM quite extensively in my work, my model of how accelerate codegen works is limited to what I've been able to glean from Trevor's papers, so I might be barking up the wrong tree entirely. If others think this is a good idea I'd be happy to offer to mentor a student.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.