FluxML / FluxML/Optimisers.jl

`destructure`'s gradient is confused by `trainable`

Open
#72 0 comments 0 reactions 0 assignees View on GitHub
bug gradients
Dominant language
Julia
Stars
96
Forks
30
PR merge metrics
No merged PRs in 30d

Description

Example:
```julia
using Optimisers, Functors, Zygote

struct TwoThirds a; b; c; end # from the tests
Functors.@functor TwoThirds (a, c)
Optimisers.trainable(x::TwoThirds) = (a = x.a,)

mtt = TwoThirds([1.0, 2.0], [3.0], [4.0, 5.0])
v, re = destructure(mtt)
re(100v) # TwoThirds([100.0, 200.0], [3.0], [4.0, 5.0])

gradient(mtt) do x
w, _ = destructure(x)
1000 * prod(w)
end # ((a = [2000.0, 1000.0], b = nothing, c = [4.0, 5.0]),)
```
Here `b` is correct (excluded from children), but `c` (non-trainable child) has a value not a gradient.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.