JuliaArrays / JuliaArrays/ArrayInterface.jl
Managing array memory
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 140
- Forks
- 44
- Avg merge
- 4d 14h
- Merged PRs (30d)
- 5
Description
The formal interface for working with an array's memory is currently very limited. We mostly just have ways of accessing preallocated memory (e.g., unsafe_wrap and unsafe_pointer_to_objref). The rest of the time we need to rely on methods like push!/pop!/append!/resize to mutate a Vector down the line. Likewise, if we want to allocate memory for a new array we typically need to wrap a new instance of Array.
A couple reasons we this would be good to have here
- Performance. The necessity to ensure that memory is safely managed also means that each call to something like
resize!has to ensure that we don't create a situation where we are out of bounds. This probably only has minimal overhead in most cases but if we are doing something complicated like merging and or sorting two vectors then we may be callingresize!a lot. I recall this sort of thing coming up a lot when I was trying to work with graph algorithms last year that do a lot of insertion and deletion. - Necessary for creating some interfaces. One of the things that makes the indexing interface so nice is that there are formally defined places for new types to insert and propagate information (e.g., bounds checking). It also makes it easier to optimize indexing. In contrast,
Vectordirectly uses methods that allocate and write to memory (e.g.,Base._growend!andBase.arrayset!) while abstract types typically hope thatresize!works and then usesetindex!.
From where I'm standing it seams like this would be great to have here. The reason this is an issue instead of a PR is mainly because I'm unsure if the implementation for this would be too general or involved for this package. We could always define several methods here like unsafe_grow_end! and unsafe_shrink_end! but these aren't super helpful without implementations that can interact with pointers and references.
@chriselrod, do you think growing/shrinking/allocating memory in an efficient way would require a bunch of LLVM magic or could we do it pretty simply here?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing unsafe_wrap, unsafe_pointer_to_objref, Base._growend!, and Base.arrayset! interfaces and how resize! is used for vectors. Determine whether efficient growing, shrinking, and allocation can be exposed with a sufficiently general interface, then document a concrete scope and implementation path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100