JuliaDiff / JuliaDiff/ForwardDiff.jl
confused by this root finding example
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 1k
- Forks
- 160
- PR merge metrics
- No merged PRs in 30d
Description
I have some code which calls a root finding step, via the Roots package that I'd like to be able to differentiate. It appears that whether ForwardDiff "works" in this setting depends on some subtlety that I don't quite understand.
For an MWE, consider finding the (unique) root of the cubic function f(x, k) = (x-k)^3 which has a root at x = k. Let g1(k) = find_zero(x -> f(x, k), [k-1.0,k+1.0]) and similarly let g2(k) = find_zero(x -> f(x, k), [10.0,+10.0]). Note that for values of k between -10 to +10, g1(k) == g2(k). In this range, the only difference between g1 and g2 is that the root finding step in g1 is "aware" of the value of k to look for, via the bounds I have provided, whereas it is ignorant of them in g2. For values of k in this range, the derivatives of g1 and g2 are identical, and equal to 1.0. However, I am seeing that ForwardDiff thinks the derivative of g2 is zero, while it finds the derivative of g1 to be 1.0:
Using Roots, ForwardDiff, FiniteDiff
g0(x,k) = (x-k)^3
g1(k) = find_zero(x -> g0(x,k), [-10.0, +10.0])
g2(k) = find_zero(x -> g0(x,k), [-1.0 * k, +1.0 * k])
Julia reports that the functions are the same, and do vary with the input in the expected way:
ulia> g1(3)
3.0
julia> g2(3)
3.0
julia> g1(4)
4.0
julia> g2(4)
4.0
However, the ForwardDiff gradients are different:
julia> ForwardDiff.derivative(g1, 3.0)
0.0
julia> ForwardDiff.derivative(g2, 3.0)
1.0
Note that FiniteDiff works fine here (up to numerical approximation):
julia> FiniteDiff.finite_difference_derivative(g1, 3.0)
0.9999999999991082
julia> FiniteDiff.finite_difference_derivative(g2, 3.0)
0.9999999999991082
Thanks in advance for any suggestions the ForwardDiff team has in understanding this difference.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the MWE entry points g1, g2, and ForwardDiff.derivative using the Roots, ForwardDiff, and FiniteDiff calls shown in the issue. Compare how the two find_zero bounds affect differentiation and determine whether the differing derivatives are expected; done means documenting the cause or identifying a focused correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100