JuliaParallel / JuliaParallel/DistributedArrays.jl
Unnecessary memory allocations in restrict_indices
- Dominant language
- Julia
- Stars
- 205
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I found this when I was puzzled by a [discourse question](https://discourse.julialang.org/t/strange-memory-allocation-with-distributedarrays/38325).
The function `restrict_indices` (called on line 824 in darray.jl, for example, couldn't figure out how to reference here), allocates an array of indices as big as the array that that you are trying to copy. Thus doubling the memory allocations.
Not sure if there is a way around it (just allocating tuples of ranges of indices or the like), but I thought I should file an issue so it doesn't get lost.
Cheers!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading restrict_indices and its call at line 824 of darray.jl, then inspect how the index array is created during copying. Confirm the reported allocation behavior from the linked Discourse question. Done means avoiding the unnecessary array-sized allocation while preserving restrict_indices behavior and distributed array copying.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100