JuliaArrays / JuliaArrays/AxisArrays.jl
Stackoverflow error when doing cat on array of AxisArrays
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 207
- Forks
- 41
- PR merge metrics
- No merged PRs in 30d
Description
Using AxisArrays v0.4.4, I encountered a StackOverflow error when doing concatenation with a relatively large array as below:
stackoverflowarray = [AxisArray(rand(500,4)) for i in 1:1048]
cat(stackoverflowarray...; dims=3)
This is actually not an uncommon size in MCMC methods, since 500x4 is just for 4 chains with 500 samples each, and 1048 would be the number of parameters.
Additionally, while doing the cat operation, I will see my RAM consistently climb for about 6GB until the stackoverflow error is thrown. This is probably not the cause though, but it might be correlated/side-effect, because I still have more RAM available, since I have seen julia climb to 12GB at times, and now it is at a total of 7GB when it throws the error.
Lastly, note that if I just do this:
nostackoverflowarray = [AxisArray(rand(500,4)) for i in 1:1048]
cat(nostackoverflowarray...; dims=3)
There is no stackoverflow errors, it finishes in a second, and the operation consumes maybe 25 MB
StackoverflowError:
ERROR: StackOverflowError: Stacktrace: [1] _zip_iterate_all(::NTuple{1048,Tuple{Int64,Int64}}, ::NTuple{1048,Tuple{}}) at ./iterators.jl:341 [2] iterate(::Base.Iterators.Zip{NTuple{1048,Tuple{Int64,Int64}}}) at ./iterators.jl:334 [3] sizes(::AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}}, ::Vararg{AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}},N} where N) at /home/michiel/.julia/packages/AxisArrays/FWWEV/src/combine.jl:12 [4] matchingdims(::NTuple{1048,AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}}}) at /home/michiel/.julia/packages/AxisArrays/FWWEV/src/combine.jl:13 [5] _cat(::Int64, ::AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}}, ::Vararg{AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}},N} where N) at /home/michiel/.julia/packages/AxisArrays/FWWEV/src/combine.jl:26 [6] cat(::AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}}, ::AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}}, ::Vararg{AxisArray{Float64,2,Array{Float64,2},Tuple{Axis{:row,Base.OneTo{Int64}},Axis{:col,Base.OneTo{Int64}}}},N} where N; dims::Int64) at /home/michiel/.julia/packages/AxisArrays/FWWEV/src/combine.jl:16 [7] top-level scope at REPL[4]:1Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the 1,048-element AxisArray example and inspect src/combine.jl, especially sizes, matchingdims, and _cat at the stack-trace locations. Compare it with the plain-array concatenation case; done means cat completes without StackOverflowError and preserves the expected concatenated result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100