JuliaArrays / JuliaArrays/StaticArrays.jl

hcat/vcat on large arrays not returning/hanging/very slow

Open
#953 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

compile-time-performance
Dominant language
Julia
Stars
844
Forks
159
Avg merge
3d 21h
Merged PRs (30d)
3

Description

Hi, I'm having a troublesome issue with vector of SVector.
When I try to convert an Vector{SVector{3,Float64} into a Nx3 Matrix for plotting, Julia doesn't return if the vector is too long.

It seems that SVector require a lots of memory allocation for the job and 99% most of the time is used in wasted compilation time. (See example below)
Normal Vector of Vector doesn't have this issue. It is possible the issue has already been flagged I didn't find it.

Normal Vectors:

julia> v = [zeros(3) for i in 1:10_000]
10000-element Vector{Vector{Float64}}:
 [0.0, 0.0, 0.0]
 ⋮
 [0.0, 0.0, 0.0]

julia>  @time hcat(v...)
  0.000528 seconds (7 allocations: 391.000 KiB)
3×10000 Matrix{Float64}:
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0

Issue:

julia> using StaticArrays
julia> v = zeros(SVector{3,Float64}, 100)
julia> @time hcat(v...)

@time hcat(v...)
  5.367366 seconds (4.43 M allocations: 230.121 MiB, 0.62% gc time, 99.70% compilation time)
3×100 SMatrix{3, 100, Float64, 300} with indices SOneTo(3)×SOneTo(100):
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0

if size is higher than 1000 the function just doesn't return anything.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by running the reported StaticArrays reproducer with hcat(v...) for 100 and larger inputs, then trace the hcat and SVector/SMatrix dispatch involved. Done means large vectors return without the reported excessive compilation time, allocations, or hanging behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.