JuliaDiff / JuliaDiff/ReverseDiff.jl
Incorrect zero gradient with no error thrown
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 393
- Forks
- 60
- Avg merge
- 18h 24m
- Merged PRs (30d)
- 8
Description
I have am trying to take gradients of a loss function which is defined via this function below. I need to take gradients of tensor_conv w.r.t W and H, which are not mutated inside the function:
function tensor_conv!(est, W::AbstractArray, H::AbstractArray)
K, N, L = size(W)
T = size(H, 2)
@. est = 0
for lag = 0:(L-1)
@views s_dot!(est[:, lag+1:T], W[:, :, lag+1]', H, lag, 1, 1)
end
return est
end
function s_dot!(B, Wl, H, lag, α, β)
K, T = size(H)
if lag < 0
@views mul!(B, Wl, H[:, 1+lag:T], α, β)
else # lag >= 0
@views mul!(B, Wl, H[:, 1:T-lag], α, β)
end
return B
end
However, reverseDiff always gives me zero gradient when differentiating this function, it does not throw an error. Can someone explain what might be going wrong?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the zero-gradient result with tensor_conv! and s_dot! using ReverseDiff, then isolate whether the mutation, @views, or mul! call affects differentiation. Compare the computed gradients with expected values and document the cause or a minimal failing case; no repository file or test is named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100