JuliaDiff / JuliaDiff/ChainRulesCore.jl
Aliased differentials and Inplace Accumulation
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 267
- Forks
- 66
- PR merge metrics
- No merged PRs in 30d
Description
If a pullback is something like dx->(dx, dx) for dx being a reference type, e.g. like the pullback for +(::Array, ::Array), then the two outputs are the same object.
They are aliased.
Even though they are actually differentials for (usually) distinct (nonaliased) primal values.
This causes inplace accumulation to act wrong.
See discussion here https://github.com/FluxML/Zygote.jl/pull/962#issuecomment-835884201
where @mcabbott was just adding inplace accumulation for getindex.
I am wondering if we need to require that if the primals are not aliased the differentials also need to not be aliased.
This extra copies is slow if not inplace accumulation is not actually done though.
we might want to make it configurable depending on the AD system.
(this related to the config needs for #68)
I think the ideal solution would be copy-on-write objects.
But doing that without language support is suffering.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked discussion in pull request #962, especially the getindex inplace-accumulation work and the relationship to configuration issue #68. Determine the intended aliasing semantics for differentials, whether copying should be configurable, and what behavior would demonstrate that inplace accumulation is correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100