JuliaDiff / JuliaDiff/ForwardDiff.jl

Is this code a supported use? (single-pass value and derivative)

Open
#610 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
1k
Forks
160
PR merge metrics
No merged PRs in 30d

Description

Considering these two methods that compute the value and first derivative of a scalar function in a single pass:

import ForwardDiff
import DiffResults

@inline function value_and_derivative(f, Y::Type, x::Real)
    diffresult = ForwardDiff.derivative!(DiffResults.DiffResult(zero(Y), zero(Y)), f, x)
    return DiffResults.value(diffresult), DiffResults.derivative(diffresult)
end

@inline function value_and_derivative(f, x::Real)
    T = typeof(ForwardDiff.Tag(f, typeof(x)))
    ydual = f(ForwardDiff.Dual{T}(x, one(x)))
    return ForwardDiff.value(T, ydual), ForwardDiff.extract_derivative(T, ydual)
end
  • The first method seems to be the officially recommended way to do this. However, (1) it introduces the DiffResults dependency just for this, and (2) it needlessly requires the caller to specify f's return type ahead of the call. Neither are dealbreakers, but IMO they add friction for something that looks like it shouldn't have it (intuition says that the value comes for free when computing a derivative with ForwardDiff)

  • The second method uses the Tag and Dual types as well as the extract_derivative function, which are not listed in the "Differentiation API" in the docs, so I'm not sure if they're considered part of the stable public API

Both methods run equally fast (and significantly faster than a naive two-pass implementation), so my question is: does the second method constitute a supported use of ForwardDiff's public API?

If such an use isn't supported (but maybe even if it is so—both method implementations appear too convoluted for a pretty common use case IMO), I'd like to suggest adding a value_and_derivative function with the second method to either this or one of the other packages in JuliaDiff (I'm willing to write a PR).

Related: #401, #391

EDITS: y-> ydual, ydual.value -> ForwardDiff.value(T, ydual), add another related issue

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ForwardDiff's documented Differentiation API and compare it with the shown derivative!, Tag, Dual, value, and extract_derivative usage. Review related issues #401 and #391; done means clarifying whether the second approach is supported and, if not, defining the scope for a public value_and_derivative API.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
developer-experience
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.