JuliaArrays / JuliaArrays/ArrayInterface.jl
Array Constructors - `similar`
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 140
- Forks
- 44
- Avg merge
- 4d 14h
- Merged PRs (30d)
- 5
Description
I think we are in a good place to start working on array constructors (would also make documenting examples a whole lot easier). This brings up stuff related to similar:
- https://github.com/JuliaLang/julia/issues/25107
- https://github.com/JuliaLang/julia/issues/18161
- https://github.com/JuliaLang/julia/issues/10064
I'm not convinced there's a silver bullet for array constructors, but I think we could at least find a solution to what similar is often trying to do, allow generic method definition without worrying about array specific constructors. I think we actually need to break up what similar does into several pieces though
- allocate memory - a buffer/Ref that is safer than a pointer
- do something to the memory - probably fastest on pointers
- create user facing instance - e.g., Array
I haven't worked out all the details but here's some of the cleaner code I have so far that might support this.
""" allocate_memory(x, args...) """ # step 1 allocates memory with corresponding device to `x` and axes
function allocate_memory(x) end
""" preserve(f, x, y) """ # step 2 operate on pointers and permit novel garbage collection approaches
preserve(f, x::X) where {X} = preserve(f, x, device(X))
preserve(f, x::X, ::CPUIndex) where {X} = f(x)
function preserve(f, x::X, ::CPUPointer) where {X}
GC.@preserve x out = f(pointer(x))
return out
end
preserve(f, x::X, y::Y) where {X,Y} = preserve(p_x -> preserve(Base.Fix1(op, p_x), y), x)
""" as_immutable """
function as_immutable(x) end
""" initialize(original, new_data) """ # step 3 turn processed buffer into user facing array
function initialize(x::X, data) where {X}
if !ismutable(X)
return as_immutable(data)
else
return data
end
end
""" instantiate(f, x, allocator, initiializer) """
instantiate(f, x, allocator, initiializer) = initiializer(x, preserve(f, allocator(x), x))
instantiate(f, x, allocator) = (f, x, allocator, initiialize)
instantiate(f, x) = (f, x, allocate_memory)
The reason I think separating things out like this is helpful is because it turns this function from base like this
function rotr90(A::AbstractMatrix)
ind1, ind2 = axes(A)
B = similar(A, (ind2,ind1))
m = first(ind1)+last(ind1)
for i=ind1, j=axes(A,2)
B[j,m-i] = A[i,j]
end
return B
end
into this
function rotr90(A)
ind1, ind2 = axes(A)
return instantiate(A, x -> allocate_memory(x, (ind2, ind1))) do a, b
m = first(ind1)+last(ind1)
for i=ind1, j=axes(A,2)
b[j,m-i] = a[i,j]
end
end
end
This means that new array types typically wouldn't need to change rotr90 but would just change their allocators and initializers. I'm not super familiar with how jagged arrays work but we could have allocate_memory take in device and memory layout info for this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the three linked Julia issues and the proposed allocate_memory, preserve, initialize, and instantiate sketch. The issue names no repository files or tests, and done is not defined beyond reaching agreement on a design for separating array construction responsibilities.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100