JuliaDiff / JuliaDiff/ChainRules.jl
Current rules for sqrt produce NaN for zero primal and (co)tangents
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 475
- Forks
- 98
- PR merge metrics
- No merged PRs in 30d
Description
This only happens when the (co)tangent is 0.
julia> using ChainRules
julia> ChainRules.frule((ChainRules.ZeroTangent(), 0.0), sqrt, 0.0)
(0.0, NaN)
julia> ChainRules.rrule(sqrt, 0.0)[2](0.0)
(ChainRulesCore.NoTangent(), NaN)
I suggest we adopt the convention that the produced (co)tangent in this case should also be 0. This is supported by finite differerences:
julia> using FiniteDifferences
julia> jvp(central_fdm(5, 1), sqrt, (0.0, 0.0))
0.0
julia> j′vp(central_fdm(5, 1), x -> sqrt(clamp(x, 0, Inf)), 0.0, 0.0)
(0.0,)
julia> j′vp(central_fdm(5, 1), sqrt ∘ abs, 0.0, 0.0)
(0.0,)
So instead of using @scalar_rule we would explicitly define the frule and rrule.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing sqrt rule generated by @scalar_rule and inspect the frule and rrule entry points shown in the issue. Reproduce the zero-primal, zero-(co)tangent cases, then verify that both rules return a zero produced tangent while preserving the shown primal result. Use the FiniteDifferences examples as behavioral checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100