Default to sparse arrays with 64-bits indices
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 1.4k
- Forks
- 281
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 30
Description
MWE:
I = [1, 2^32]
J = [1, 2^32]
V = [1.0, 1.0]
sp = sparse(I, J, V)
spc = cu(sp) # InexactError
In my actual use case, my matrix was not even 2^32 large, it was close to 2^28, but I still get the same InexactError. Specifically:
- size: (506_255_563, 506_255_563)
- number of nonzeros: 3_384_286_971
Anyways, is there a way to force CUSAPRSE to use Int64?
Here's the traceback for InexactError:
ERROR: InexactError: trunc(Int32, 4294967296)
Stacktrace:
[1] throw_inexacterror(f::Symbol, #unused#::Type{Int32}, val::Int64)
@ Core ./boot.jl:634
[2] checked_trunc_sint
@ ./boot.jl:656 [inlined]
[3] toInt32
@ ./boot.jl:693 [inlined]
[4] Int32
@ ./boot.jl:783 [inlined]
[5] convert
@ ./number.jl:7 [inlined]
[6] setindex!
@ ./array.jl:969 [inlined]
[7] _unsafe_copyto!(dest::Vector{Int32}, doffs::Int64, src::Vector{Int64}, soffs::Int64, n::Int64)
@ Base ./array.jl:250
[8] unsafe_copyto!
@ ./array.jl:304 [inlined]
[9] _copyto_impl!
@ ./array.jl:327 [inlined]
[10] copyto!
@ ./array.jl:314 [inlined]
[11] copyto!
@ ./array.jl:339 [inlined]
[12] copyto_axcheck!
@ ./abstractarray.jl:1180 [inlined]
[13] Vector{Int32}(x::Vector{Int64})
@ Base ./array.jl:621
[14] Array
@ ./boot.jl:501 [inlined]
[15] convert
@ ./array.jl:613 [inlined]
[16] CuArray
@ ~/.julia/packages/CUDA/tVtYo/src/array.jl:359 [inlined]
[17] CuArray
@ ~/.julia/packages/CUDA/tVtYo/src/array.jl:363 [inlined]
[18] (CuSparseMatrixCSC{Float32})(Mat::SparseMatrixCSC{Float64, Int64})
@ CUDA.CUSPARSE ~/.julia/packages/CUDA/tVtYo/lib/cusparse/array.jl:365
[19] adapt_storage
@ ~/.julia/packages/CUDA/tVtYo/lib/cusparse/array.jl:418 [inlined]
[20] adapt_structure
@ ~/.julia/packages/Adapt/UtItS/src/Adapt.jl:57 [inlined]
[21] adapt
@ ~/.julia/packages/Adapt/UtItS/src/Adapt.jl:40 [inlined]
[22] adapt_storage
@ ~/.julia/packages/CUDA/tVtYo/lib/cusparse/array.jl:422 [inlined]
[23] adapt_structure
@ ~/.julia/packages/Adapt/UtItS/src/Adapt.jl:57 [inlined]
[24] adapt
@ ~/.julia/packages/Adapt/UtItS/src/Adapt.jl:40 [inlined]
[25] #cu#1022
@ ~/.julia/packages/CUDA/tVtYo/src/array.jl:664 [inlined]
[26] cu(xs::SparseMatrixCSC{Float64, Int64})
@ CUDA ~/.julia/packages/CUDA/tVtYo/src/array.jl:664
[27] top-level scope
@ REPL[23]:1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the sparse conversion with the Julia example, then inspect lib/cusparse/array.jl around the CuSparseMatrixCSC conversion and adapt_storage entry points shown in the traceback. Check how Int64 indices are converted during cu(sp); done means a sparse matrix with indices beyond Int32 converts without the reported InexactError and its indices remain usable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100