JuliaArrays / JuliaArrays/StructArrays.jl
Large number of allocations in 0.6.8
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 402
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
The eager automatic conversion introduced in #227 (cc @timholy @piever) leads to many allocations when this conversion isn't actually needed.
For example:
StructArrays@0.6.7 - 25 allocations:
julia> using StructArrays, ArraysOfArrays
julia> A = StructArray(x=VectorOfVectors{Int}())
julia> @time for _ in 1:10^6 push!(A, (x=1:10,)) end
0.307580 seconds (25 allocations: 11.269 MiB, 7.45% gc time)
StructArrays@0.6.8 - 1 million allocations:
julia> using StructArrays, ArraysOfArrays
julia> A = StructArray(x=VectorOfVectors{Int}())
julia> @time for _ in 1:10^6 push!(A, (x=1:10,)) end
0.809896 seconds (1.00 M allocations: 293.601 MiB, 55.02% gc time)
Of course, these allocations go away if automatic conversion (https://github.com/JuliaArrays/StructArrays.jl/blob/1581d70090de650646277e4eee4e33794e55b9ac/src/utils.jl#L197-L199) is disabled by StructArrays.maybe_convert_elt(::Type{T}, vals::Tuple) where {T} = vals.
This change in 0.6.8 is highly breaking performance-wise. Can the new automatic conversion approach be revised and made less eager somehow?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the allocation benchmark in the issue, comparing StructArrays 0.6.7 and 0.6.8. Inspect the automatic-conversion logic in src/utils.jl at the referenced lines and the behavior of maybe_convert_elt(::Type{T}, vals::Tuple). Done means revising the conversion behavior so the reported push! workload no longer incurs one million allocations while required conversion still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100