JuliaDiff / JuliaDiff/ReverseDiff.jl
Incorrect results when nesting ReverseDiff inside ForwardDiff
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 393
- Forks
- 60
- Avg merge
- 18h 24m
- Merged PRs (30d)
- 8
Description
I'm not sure if this is even supported, but I think it should throw an error instead of returning zero gradients
The first example is an old bug (#45 ), the second shows you can't nest ReverseDiff inside ForwardDiff, the third shows you can the other way around.
julia> let
D(f, x) = ReverseDiff.gradient(x->f(x[1]), [x])[1]
u1 = D(x -> x * D(y -> x * y, 3), 5) # 3
u2 = D(x -> x * D(y -> y * x, 3), 5) # 5
@show u1, u2
end
(u1, u2) = (3, 5)
(3, 5)
julia> let
Dr(f, x) = ReverseDiff.gradient(x->f(x[1]), [x])[1]
Df(f, x) = ForwardDiff.gradient(x->f(x[1]), [x])[1]
u1 = Df(x -> x * Dr(y -> x * y, 3), 5)
u2 = Df(x -> x * Dr(y -> y * x, 3), 5)
@show u1, u2
end
(u1, u2) = (0, 0)
(0, 0)
julia> let
Dr(f, x) = ReverseDiff.gradient(x->f(x[1]), [x])[1]
Df(f, x) = ForwardDiff.gradient(x->f(x[1]), [x])[1]
u1 = Dr(x -> x * Df(y -> x * y, 3), 5)
u2 = Dr(x -> x * Df(y -> y * x, 3), 5)
@show u1, u2
end
(u1, u2) = (10, 10)
(10, 10)
(test3) pkg> status
Status `~/Sandboxes/Julia-Misc/test3/Project.toml`
[f6369f11] ForwardDiff v0.10.16 `~/.julia/dev/ForwardDiff`
[37e2e3b7] ReverseDiff v1.5.0 `~/.julia/dev/ReverseDiff`
julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: AMD Ryzen 7 3700X 8-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, znver2)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the nested calls using the ReverseDiff.gradient and ForwardDiff.gradient entry points shown in the issue with the listed Julia and package versions. Determine the intended behavior for ReverseDiff inside ForwardDiff, then add coverage for the reported zero-gradient case and verify that the resulting behavior is explicit and consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100