JuliaGaussianProcesses / JuliaGaussianProcesses/KernelFunctions.jl

ForwardDiff fails for Matern kernels when the two arguments are equal

Open
#571 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
275
Forks
41
PR merge metrics
No merged PRs in 30d

Description

I have a project in which I take gradients of Matern kernels with respect to an argument via ForwardDiff and was able to do so without issue until recently. When using KernelFunctions v0.10.58 or greater or ForwardDiff v1.0.0 or greater, however, calling ForwardDiff.gradient with respect to an argument on a Matern kernel returns a vector of NaNs when the two arguments to the kernel are equal. If I use Zygote instead I get a vector of zeros. MWE:

using KernelFunctions 
using ForwardDiff 
using Zygote 

vec1 = randn(2) 
vec2 = randn(2) 

myfun(x) = MaternKernel(ν=2.1)(x, vec1) 

ForwardDiff.gradient(myfun, vec1) # vector of NaNs
ForwardDiff.gradient(myfun, vec2) # works as expected 

Zygote.gradient(myfun, vec1) # vector of 0s
Zygote.gradient(myfun, vec2) # works as expected 

For reference, ForwardDiff.gradient(myfun, vec1) returns a vector of zeros with KernelFunctions v0.10.57 and ForwardDiff v0.10.38.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided MWE with KernelFunctions v0.10.58 or later and ForwardDiff v1.0.0 or later, focusing on ForwardDiff.gradient for MaternKernel when both arguments are equal. Compare the result with unequal arguments, older package versions, and Zygote; done means the equal-argument gradient no longer returns NaNs while the existing cases continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.