JuliaDiff / JuliaDiff/ReverseDiff.jl

ReverseDiff documentation shows issue that has been fixed? Nested differentiation of a closure?

Open
#222 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
393
Forks
60
Avg merge
18h 24m
Merged PRs (30d)
8

Description

Hello everyone. The ReverseDiff documentation under "Limitations of ReverseDiff" lists the following

Nested differentiation of closures is dangerous. Differentiating closures is safe, and nested differentation is safe, but you might be vulnerable to a subtle bug if you try to do both. See this ForwardDiff issue for details. A fix is currently being planned for this problem.

But it appears as the listed ForwardDiff issue has been resolved. I ran this simple test

ReverseDiff.gradient(x -> x .+ ReverseDiff.gradient(y -> x + 2*y,[1]), [1]) # Result=1

Which results in the correct value of 1. It seems as though this issue has been fixed?

As a bit of background, I have been working hard to use a hessian in a Flux loss function (which requires me to take the gradient of this hessian wrt a NN model's weights), but Zygote has a large number of issues with nested differentiation at the moment (not docking them at all, its an amazing development). It appears as though I am able to use ReverseDiff to correctly take the gradient of the hessian wrt the weights using the Flux.destructure function

using Flux,ReverseDiff,ForwardDiff

model = Chain(Dense(4=>2,σ),Dense(2=>1,σ)) # construct the model
weights,reconstructor = Flux.destructure(model) # destructure the model so ReverseDiff can track the weights
x = rand(4) # make a random data point

# calculate the gradient of the first element of the hessian wrt the model's weights
ReverseDiff.gradient( temp -> ForwardDiff.hessian(temp2 -> reconstructor(temp)(temp2)[1],x)[1] , weights) 

I am able to use these gradients to train the model, and it appears to be working. I'm worried that the gradients may be only slightly incorrect, but close enough for the optimizer to still work. If someone could comment on the status of this issue, I would greatly appreciate it.

Thanks in advance!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ReverseDiff documentation section "Limitations of ReverseDiff" and reproduce the nested-closure example from the issue. Compare its warning with the linked ForwardDiff issue and the provided ReverseDiff and Flux examples. Done means confirming the limitation's current status and updating the documentation if its wording or link is outdated.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.