TuringLang / TuringLang/DynamicPPL.jl
Known issues with VNT, part 3 -- multiindex-upon-multiindex can give wrong keys with ArrayLikeBlocks
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 286
- Forks
- 41
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 34
Description
using DynamicPPL
using DynamicPPL: templated_setindex!!
struct A end
Base.size(::A) = (2,)
vnt = VarNamedTuple()
vnt = templated_setindex!!(vnt, A(), @varname(x[2:4][1:2]), randn(4))
keys(vnt)
# 1-element Vector{VarName}:
# x[1:2]
Here, we are setting indices 2 and 3 of the parent array x. However, keys(vnt) reports [1:2].
The reason for this is because the index in the key is obtained from the ArrayLikeBlock that is used to store the sized thing. In turn, the ArrayLikeBlock's indices are obtained from the indices that were used to initially set it inside the array, which were 1:2 -- it has no way of knowing that there was a 2:4 that came before it.
If this was x[1:N][P:Q] this would actually work perfectly fine for any N,P,Q but this issue is a worst-case scenario where the first set of multiindices don't begin at 1.
The fact that the stored indices are wrong can also lead to weird results when merging two VNTs with data that is mislabelled like this.
Note that the ALBs are still stored in the correct position in the PartialArray (indices 2 and 3 are the actual thing that we stored -- the other two indices are junk data generated via undef, but are masked so will never be seen). It's just the keys that are weird.
julia> vnt.data.x.data
4-element Vector{DynamicPPL.VarNamedTuples.ArrayLikeBlock{A, Tuple{UnitRange{Int64}}, @NamedTuple{}, Tuple{Int64}}}:
ArrayLikeBlock(A(), (4526817440:4526817488,), NamedTuple(), (4526817536,))
ArrayLikeBlock(A(), (1:2,), NamedTuple(), (2,))
ArrayLikeBlock(A(), (1:2,), NamedTuple(), (2,))
ArrayLikeBlock(A(), (0:0,), NamedTuple(), (131330,))
(This is good: it means that if we store an array instead of an ALB, the keys will be correctly picked up, because they are in the right positions.)
vnt = VarNamedTuple()
vnt = templated_setindex!!(vnt, randn(2), @varname(x[2:4][1:2]), randn(4))
keys(vnt)
# 2-element Vector{VarName}:
# x[2]
# x[3]
I personally think this is niche enough that we shouldn't need to care about it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the example with templated_setindex!!, @varname(x[2:4][1:2]), and an ArrayLikeBlock, then inspect how ArrayLikeBlock indices and PartialArray positions are used to form keys. Done means keys(vnt) reports the actual stored positions, such as x[2] and x[3], rather than x[1:2], without disrupting the correctly positioned data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100