JuliaDiff / JuliaDiff/ReverseDiff.jl

Error with Turing and ReverseDiff, adding matrices

Open
#175 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
393
Forks
60
Avg merge
18h 24m
Merged PRs (30d)
8

Description

Hi -

I've been using ReverseDiff to run some models in Turing, and it appears a recent change from v1.7 to 1.8 has made my models stop working. I managed to narrow down the issue to a single line in the model, where I try to add two matrices. Oddly, it only seems to be an issue with particular combinations of matrices, but I'm not seeing the pattern, so I'm not sure what the problem is.

See MWE below:

using Turing, ReverseDiff

@model function efa(P)
    λ ~ filldist(Normal(0, 10), P, 1)
    Ψ ~ filldist(truncated(Normal(0, 5), 0, Inf), P)
    f_var ~ filldist(truncated(Normal(0, 5), 0, Inf), 1)

    Σ1 = λ * f_var * λ' + Diagonal(Ψ)        # Fails
    # Σ1 = λ * f_var * λ' + diagm(Ψ)           # Fails
    # Σ1 = randn(P, P) + Diagonal(Ψ)           # Works
    # Σ1 = λ * f_var * λ' + randn(P, P)        # Works
    # Σ1 = λ * f_var * λ' + Diagonal(ones(P))  # Works
end

Turing.setadbackend(:reversediff)
efa_test = efa(3)
@time m1 = sample(efa_test, NUTS(1, 0.65), MCMCThreads(), 1, 4)

Running this gives an error like this (only the first few lines of the stacktrace shown):

nested task error: MethodError: no method matching increment_deriv!(::Float64, ::Float64)
        Closest candidates are:
          increment_deriv!(::ReverseDiff.TrackedArray, ::Real, ::Any) at C:\Users\opus\.julia\packages\ReverseDiff\60noS\src\derivatives\propagation.jl:34
          increment_deriv!(::AbstractArray, ::Real, ::Any) at C:\Users\opus\.julia\packages\ReverseDiff\60noS\src\derivatives\propagation.jl:36    
          increment_deriv!(::AbstractArray, ::Any) at C:\Users\opus\.julia\packages\ReverseDiff\60noS\src\derivatives\propagation.jl:38
          ...
        Stacktrace:
          [1] increment_deriv!
            @ ~\.julia\packages\ReverseDiff\60noS\src\derivatives\propagation.jl:35 [inlined]
          [2] increment_deriv!(t::Matrix{Real}, x::Matrix{Float64})
            @ ReverseDiff ~\.julia\packages\ReverseDiff\60noS\src\derivatives\propagation.jl:40
          [3] special_reverse_exec!(instruction::ReverseDiff.SpecialInstruction{typeof(+), Tuple{ReverseDiff.TrackedArray{Float64, Float64, 2, Matrix{Float64}, Matrix{Float64}}, Matrix{Real}}, ReverseDiff.TrackedArray{Float64, Float64, 2, Matrix{Float64}, Matrix{Float64}}, Nothing})
            @ ReverseDiff ~\.julia\packages\ReverseDiff\60noS\src\derivatives\linalg\arithmetic.jl:50
          [4] reverse_exec!(instruction::ReverseDiff.SpecialInstruction{typeof(+), Tuple{ReverseDiff.TrackedArray{Float64, Float64, 2, Matrix{Float64}, Matrix{Float64}}, Matrix{Real}}, ReverseDiff.TrackedArray{Float64, Float64, 2, Matrix{Float64}, Matrix{Float64}}, Nothing})
            @ ReverseDiff ~\.julia\packages\ReverseDiff\60noS\src\tape.jl:93
          [5] reverse_pass!(tape::Vector{ReverseDiff.AbstractInstruction})
            @ ReverseDiff ~\.julia\packages\ReverseDiff\60noS\src\tape.jl:87
          [6] reverse_pass!
            @ ~\.julia\packages\ReverseDiff\60noS\src\api\tape.jl:36 [inlined]

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the Julia MWE and inspect the failing matrix-addition path in ReverseDiff/src/derivatives/propagation.jl and src/derivatives/linalg/arithmetic.jl, especially increment_deriv! and special_reverse_exec!. Compare the working and failing matrix combinations, then verify that the Turing model samples successfully with ReverseDiff without the reported MethodError.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.