JuliaGPU / JuliaGPU/CUDA.jl

Use of array wrappers & unions regress load time

Open
#453 4 comments 0 reactions 0 assignees View on GitHub
cuda array upstream
Dominant language
Julia
Stars
1.4k
Forks
281
Avg merge
1d 7h
Merged PRs (30d)
30

Description

Previously we were using a `CuArray` type that could represent a view, reshape, reinterpret, etc. For the sake of simplicity, I switched to a simpler `CuArray` type while reusing `Base.SubArray`, `Base.ReshapeArray`, etc. That requires use of type unions to, e.g., represent all dense or strided `CuArray`s: https://github.com/JuliaGPU/CUDA.jl/blob/75f7d309b6d2b757dcdbe901049d89296b9284a7/src/array.jl#L146-L164

These definitions are [almost identical](https://github.com/JuliaLang/julia/blob/b89c4a3cb539ed47eca651cf290a20adde552fc2/base/reinterpretarray.jl#L50-L59) to how Base defines `StridedArray`. However, using them significantly regresses load time. For example, https://github.com/JuliaGPU/CUDA.jl/pull/450 adds them to a bunch of `LinearAlgebra.mul!` methods which badly affects time of `using CUDA`: +25%, https://speed.juliagpu.org/timeline/#/?exe=4&ben=latency/import&env=1&revs=50&base=3+96&equid=off&quarts=on&extr=on

In a similar vein, Adapt.jl defines a union that captures all array instances that can be used on the GPU (i.e. not necessarily dense or strided, but an `Adjoint` or `PermuteDimsArray`): https://github.com/JuliaGPU/Adapt.jl/blob/11d96a531cb70359e88ed2ad0d0a13a85727a204/src/wrappers.jl#L73-L92
Using these unions makes load time go crazy, e.g. with `mul!(::CuArray, ::AnyCuArray...)` (where `AnyCuArray` uses the `Adapt.WrappedArray` union) it goes from 5 to 25s.

I can understand how the large union from Adapt.jl is needlessly taxing on inference, and I guess we may need something like an `AbstractWrappedArray` here (https://github.com/JuliaLang/julia/pull/31563). However, with `StridedCuArray` I had not expected these regressions, as Base uses similar patterns. Am I doing anything especially bad here? I'd like to start using `StridedCuArray` much more, in order to cover APIs that take `stride` inputs (which there are quite some).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.