JuliaDiff / JuliaDiff/ForwardDiff.jl

confused by this root finding example

Open
#595 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
1k
Forks
160
PR merge metrics
No merged PRs in 30d

Description

I have some code which calls a root finding step, via the Roots package that I'd like to be able to differentiate. It appears that whether ForwardDiff "works" in this setting depends on some subtlety that I don't quite understand.

For an MWE, consider finding the (unique) root of the cubic function f(x, k) = (x-k)^3 which has a root at x = k. Let g1(k) = find_zero(x -> f(x, k), [k-1.0,k+1.0]) and similarly let g2(k) = find_zero(x -> f(x, k), [10.0,+10.0]). Note that for values of k between -10 to +10, g1(k) == g2(k). In this range, the only difference between g1 and g2 is that the root finding step in g1 is "aware" of the value of k to look for, via the bounds I have provided, whereas it is ignorant of them in g2. For values of k in this range, the derivatives of g1 and g2 are identical, and equal to 1.0. However, I am seeing that ForwardDiff thinks the derivative of g2 is zero, while it finds the derivative of g1 to be 1.0:

Using Roots, ForwardDiff, FiniteDiff
g0(x,k) = (x-k)^3
g1(k) = find_zero(x -> g0(x,k), [-10.0, +10.0])
g2(k) = find_zero(x -> g0(x,k), [-1.0 * k, +1.0 * k])

Julia reports that the functions are the same, and do vary with the input in the expected way:

ulia> g1(3)
3.0

julia> g2(3)
3.0

julia> g1(4)
4.0

julia> g2(4)
4.0

However, the ForwardDiff gradients are different:

julia> ForwardDiff.derivative(g1, 3.0)
0.0

julia> ForwardDiff.derivative(g2, 3.0)
1.0

Note that FiniteDiff works fine here (up to numerical approximation):

julia> FiniteDiff.finite_difference_derivative(g1, 3.0)
0.9999999999991082

julia> FiniteDiff.finite_difference_derivative(g2, 3.0)
0.9999999999991082

Thanks in advance for any suggestions the ForwardDiff team has in understanding this difference.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the MWE entry points g1, g2, and ForwardDiff.derivative using the Roots, ForwardDiff, and FiniteDiff calls shown in the issue. Compare how the two find_zero bounds affect differentiation and determine whether the differing derivatives are expected; done means documenting the cause or identifying a focused correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.