JuliaDiff / JuliaDiff/ChainRulesTestUtils.jl

Tests for @non_differentiable

Open
#202 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
50
Forks
18
PR merge metrics
No merged PRs in 30d

Description

It's ocassionally not possible to test @non_differentiable using test_rrule because finite differencing gives the wrong / unreliable results.

For example, this is the case in the Vector{Floa64}(undef, 5) constructor implemented in https://github.com/JuliaDiff/ChainRules.jl/pull/491 -- in this case, the values of the array produced are unreliable, so finite difference estimates of the gradient aren't consistent between different calls to the function.

One option would be to define a test_non_differentiable suite that looks something like

test_non_differentiable(f, args...; use_finite_differencing=true)

in which we prescribe a set of standard tests. In the case that the user sets use_finite_differencing=false, we could just ensure that f does indeed return NoTangents, in addition to standard things like ensuring that the primal agrees with the first output of the rrule being tested.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing test_rrule suite and the issue's proposed test_non_differentiable(f, args...; use_finite_differencing=true) interface. Define the expected standard checks from the description, including primal agreement with the rrule output and NoTangents when finite differencing is disabled. Done means the suite can test cases such as the unreliable Vector{Floa64}(undef, 5) constructor without relying on inconsistent finite differences.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.