JuliaDiff / JuliaDiff/ChainRules.jl
rrule for casting LinearAlgebra.QRCompactWYQ into a Matrix
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 475
- Forks
- 98
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to work with the Q matrix from a qr-factorization within Zygote. In an incomplete QR factorization for m>=n,
the QRCompactWYQMatrix Q has size=(m,m) but only the first n columns are relevant: https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#LinearAlgebra.qr
Using this in Zygote behaves as follows:
using LinearAlgebra
using Zygote
using Random
Random.seed!(1234)
V = rand(Float32, (6,4))
Q, _ = qr(V)
Zygote.gradient(A -> sum(A), Matrix(Q))
Zygote.gradient(A -> sum(A), Q)
Zygote.gradient(A -> sum(Matrix(A)), Q)
ERROR: LoadError: DimensionMismatch("variable with size(x) == (6, 6) cannot have a gradient with size(dx) == (6, 4)")
Stacktrace:
[1] (::ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:element, :axes), Tuple{ChainRulesCore.ProjectTo{Float32, NamedTuple{(), Tuple{}}}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}}})(dx::FillArrays.Fill{Float32, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}})
@ ChainRulesCore ~/.julia/packages/ChainRulesCore/Voykb/src/projection.jl:209
[2] Array_pullback
@ ~/.julia/packages/ChainRules/qd40H/src/rulesets/Base/array.jl:9 [inlined]
[3] ZBack
@ ~/.julia/packages/Zygote/nsu1Y/src/compiler/chainrules.jl:140 [inlined]
[4] Pullback
@ ~/source/gpuplayground/src/qr_project_mwe.jl:16 [inlined]
[5] (::Zygote.var"#50#51"{typeof(∂(#5))})(Δ::Float32)
@ Zygote ~/.julia/packages/Zygote/nsu1Y/src/compiler/interface.jl:41
[6] gradient(f::Function, args::LinearAlgebra.QRCompactWYQ{Float32, Matrix{Float32}})
@ Zygote ~/.julia/packages/Zygote/nsu1Y/src/compiler/interface.jl:76
[7] top-level scope
The first two calls to gradient work fine. The last call fails because the incoming gradient is size=(6,4) but the matrix is of size=(6,6).
I tried modifying the rrule from here, https://github.com/JuliaDiff/ChainRules.jl/blob/master/src/rulesets/Base/array.jl line 7 like this:
function ChainRules.rrule(::typeof(Matrix), x::LinearAlgebra.QRCompactWYQ)
project_x = ProjectTo(x)
Array_pullback(ȳ) = (NoTangent(), project_x(ȳ))
return T(x), Array_pullback
end
but that didn't work. Is there a good way of making calls to Matrix(Q) in a backwards pass?
@mcabbott
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduction using Matrix(Q) and Q from an incomplete qr factorization, then inspect src/rulesets/Base/array.jl and the existing Matrix rrule. Trace how ProjectTo and Array_pullback handle the (6,4) incoming gradient versus QRCompactWYQ's (6,6) size. Done means the backward pass for Matrix(Q) completes without DimensionMismatch and the existing gradient calls still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100