JuliaGPU / JuliaGPU/Metal.jl

Improve performance of Cartesian indexing

Open
#101 3 comments 0 reactions 0 assignees View on GitHub
kernels performance
Dominant language
Julia
Stars
463
Forks
68
Avg merge
1d 19m
Merged PRs (30d)
32

Description

Metal GPUs suffer from the way we encode Cartesian indices, presumably because of the integer division that happens when mapping a linear index to a Cartesian, but there may be other causes. In https://github.com/JuliaGPU/Metal.jl/pull/100 and https://github.com/JuliaGPU/GPUArrays.jl/pull/454, we worked around some of the more egregious performance issues by putting the indices in the type domain such that are known to LLVM, allowing the back-end compiler to optimize code (again, presumably avoiding the division by a constant integer by mapping them onto a bunch of bit operations).

This isn't ideal because it results in significantly more kernels being compiled. Ideally we figure out a way to better encode Cartesian indices, although it's obviously hard to avoid the integer division at all.

Alternatively, we might want to improve https://github.com/maleadt/StaticCartesian.jl, or something similar, so that we can perform this optimization ourselves instead of relying on the Metal back-end compiler, because relying on such an optimization might be fragile (as observed in https://github.com/JuliaGPU/GPUArrays.jl/pull/454 where we needed additional bounds information for the optimization to trigger).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.