JuliaDiff / JuliaDiff/ChainRules.jl

Current rules for sqrt produce NaN for zero primal and (co)tangents

Open
#576 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Julia
Stars
475
Forks
98
PR merge metrics
No merged PRs in 30d

Description

This only happens when the (co)tangent is 0.

julia> using ChainRules

julia> ChainRules.frule((ChainRules.ZeroTangent(), 0.0), sqrt, 0.0)
(0.0, NaN)

julia> ChainRules.rrule(sqrt, 0.0)[2](0.0)
(ChainRulesCore.NoTangent(), NaN)

I suggest we adopt the convention that the produced (co)tangent in this case should also be 0. This is supported by finite differerences:

julia> using FiniteDifferences

julia> jvp(central_fdm(5, 1), sqrt, (0.0, 0.0))
0.0

julia> j′vp(central_fdm(5, 1), x -> sqrt(clamp(x, 0, Inf)), 0.0, 0.0)
(0.0,)

julia> j′vp(central_fdm(5, 1), sqrt ∘ abs, 0.0, 0.0)
(0.0,)

So instead of using @scalar_rule we would explicitly define the frule and rrule.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing sqrt rule generated by @scalar_rule and inspect the frule and rrule entry points shown in the issue. Reproduce the zero-primal, zero-(co)tangent cases, then verify that both rules return a zero produced tangent while preserving the shown primal result. Use the FiniteDifferences examples as behavioral checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.