JuliaDiff / JuliaDiff/ForwardDiff.jl

Are second derivatives (and higher) inefficient?

Open
#149 9 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
1k
Forks
160
PR merge metrics
No merged PRs in 30d

Description

I'm using the Duals implementation in ForwardDiff and have a question about taking higher derivatives.

Consider this simple example:

using ForwardDiff
f(x) = sin(x) / x

@show y0 = f(10.0) # function
@show y1 = f(Dual(10.0, 1.0))  # function + 1st derivative
@show y2 = f(Dual(Dual(10.0, 1.0), Dual(1.0, 1.0))) # function + 1st,2nd derivatives

The 2nd derivative case seems unnecessarily inefficient because I must substitute a Dual number for both the value and partial components of the 1st (i.e. outer) Dual. If you run the code, you can see that y2.value.partials[1] == y2.partials[1].value, and that both give the value of 1st derivative. Must we compute this value twice? Obviously, this problem is more serious when one has a vector of derivatives to be evaluated, etc.

A modest proposal: Could we allow the types of the value and partials be different? That would seem to allow code like the following:

y2 = f(Dual(10.0, Dual(1.0, 1.0)))

Is there already some support for this, and I'm just missing it?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Julia examples using ForwardDiff.Dual and compare the repeated first-derivative values in y2.value.partials[1] and y2.partials[1]. Review the Dual implementation and determine whether heterogeneous value and partial types can avoid that duplication; done means documenting the supported approach or defining a concrete, validated change.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.