JuliaDiff / JuliaDiff/ForwardDiff.jl
Are second derivatives (and higher) inefficient?
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 1k
- Forks
- 160
- PR merge metrics
- No merged PRs in 30d
Description
I'm using the Duals implementation in ForwardDiff and have a question about taking higher derivatives.
Consider this simple example:
using ForwardDiff
f(x) = sin(x) / x
@show y0 = f(10.0) # function
@show y1 = f(Dual(10.0, 1.0)) # function + 1st derivative
@show y2 = f(Dual(Dual(10.0, 1.0), Dual(1.0, 1.0))) # function + 1st,2nd derivatives
The 2nd derivative case seems unnecessarily inefficient because I must substitute a Dual number for both the value and partial components of the 1st (i.e. outer) Dual. If you run the code, you can see that y2.value.partials[1] == y2.partials[1].value, and that both give the value of 1st derivative. Must we compute this value twice? Obviously, this problem is more serious when one has a vector of derivatives to be evaluated, etc.
A modest proposal: Could we allow the types of the value and partials be different? That would seem to allow code like the following:
y2 = f(Dual(10.0, Dual(1.0, 1.0)))
Is there already some support for this, and I'm just missing it?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Julia examples using ForwardDiff.Dual and compare the repeated first-derivative values in y2.value.partials[1] and y2.partials[1]. Review the Dual implementation and determine whether heterogeneous value and partial types can avoid that duplication; done means documenting the supported approach or defining a concrete, validated change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100