SciML / SciML/ComponentArrays.jl

Problem with `Zygote.hessian`

Open
#228 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
375
Forks
42
Avg merge
7h 25m
Merged PRs (30d)
17

Description

julia> using ComponentArrays, Zygote

julia> p = (; a = ones(2), b = zeros(2)) |> ComponentArray
ComponentVector{Float64}(a = [1.0, 1.0], b = [0.0, 0.0])

julia> Zygote.gradient(x -> sum(x.a), p)
((a = [1.0, 1.0], b = [0.0, 0.0]),)

julia> Zygote.hessian(x -> sum(x.a), p)
ERROR: type Array has no field a
Stacktrace:
  [1] adjoint
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/lib.jl:229 [inlined]
  [2] _pullback
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:66 [inlined]
  [3] #19
    @ Zygote ./REPL[12]:1 [inlined]
  [4] _pullback(ctx::Zygote.Context{false}, f::var"#19#20", args::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
  [5] pullback(f::Function, cx::Zygote.Context{false}, args::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:44
  [6] pullback(f::Any, cx::ZygoteRules.AContext, args::Vararg{Any})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:42 [inlined]
  [7] gradient(f::Function, args::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:96
  [8] (::Zygote.var"#125#126"{var"#19#20"})(x::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:64
  [9] forward_jacobian(f::Zygote.var"#125#126"{var"#19#20"}, x::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{(a = 1:2, b = 3:4)}}}, ::Val{4})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:29
 [10] forward_jacobian(f::Function, x::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{(a = 1:2, b = 3:4)}}}; chunk_threshold::Int64)
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:44
 [11] forward_jacobian
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:42 [inlined]
 [12] hessian_dual
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:64 [inlined]
 [13] hessian(f::Function, x::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{(a = 1:2, b = 3:4)}}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:62
 [14] top-level scope
    @ REPL[12]:1
 [15] top-level scope
    @ ~/.julia/packages/CUDA/nbRJk/src/initialization.jl:205

julia> Zygote.hessian_reverse(x -> sum(x.a), p)                               
ERROR: Mutating arrays is not supported -- called setindex!(Vector{Float64}, ...)
This error occurs when you ask Zygote to differentiate operations that change
the elements of arrays in place (e.g. setting values with x .= ...)               
                                                                                                                                                                     
Possible fixes:                                                                                                                                                      
- avoid mutating operations (preferred)                                                                                                                              
- or read the documentation and solutions for this error                                                                                                             
  https://fluxml.ai/Zygote.jl/latest/limitations                        
                                         
Stacktrace:                                                                                                                                                          
  [1] error(s::String)                                                                                                                                               
    @ Base ./error.jl:35                                                          
  [2] _throw_mutation_error(f::Function, args::Vector{Float64})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/array.jl:88                   
  [3] (::Zygote.var"#555#556"{Vector{Float64}})(::Nothing)                 
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/array.jl:100        
  [4] (::Zygote.var"#2655#back#557"{Zygote.var"#555#556"{Vector{Float64}}})(Δ::Nothing)                                                                                  @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71                
  [5] (::Zygote.var"#291#292"{Tuple{Tuple{Nothing, Nothing}, Tuple{Nothing}}, Zygote.var"#2655#back#557"{Zygote.var"#555#556"{Vector{Float64}}}})(Δ::Nothing)        
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/lib.jl:206
  [6] (::Zygote.var"#2169#back#293"{Zygote.var"#291#292"{Tuple{Tuple{…}, Tuple{…}}, Zygote.var"#2655#back#557"{Zygote.var"#555#556"{…}}}})(Δ::Nothing)
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71                                                                                                   
  [7] macro expansion                                                             
    @ Zygote ~/.julia/dev/ComponentArrays/src/array_interface.jl:0 [inlined]                                                                                         
  [8] _setindex!                                                                  
    @ Zygote ~/.julia/dev/ComponentArrays/src/array_interface.jl:131 [inlined]
  [9] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Nothing)                         
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0                                                                                             
 [10] setproperty!                                                                
    @ ComponentArrays ~/.julia/dev/ComponentArrays/src/namedtuple_interface.jl:17 [inlined]                                                                          
 [11] (::ComponentArrays.var"#getproperty_adjoint#87"{ComponentVector{Float64, Vector{…}, Tuple{…}}, Symbol})(Δ::FillArrays.Fill{Float64, 1, Tuple{Base.OneTo{…}}})
    @ ComponentArrays ~/.julia/dev/ComponentArrays/src/compat/chainrulescore.jl:4 [inlined]
 [12] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Tuple{Nothing, Vector{…}, Nothing})                                                
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0                                                                                             
 [13] ZBack                                                                       
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/chainrules.jl:211 [inlined]
 [14] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Tuple{Nothing, Vector{…}, Nothing})                                                                       
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
 [15] #37                                                                         
    @ Zygote ./REPL[22]:1 [inlined]                                               
 [16] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Tuple{Nothing, Vector{…}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
 [17] #75
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:45 [inlined]
 [18] (::Zygote.Pullback{Tuple{Zygote.var"#75#76"{…}, Float64}, Tuple{Zygote.Pullback{…}, Zygote.var"#2180#back#303"{…}, Zygote.Pullback{…}}})(Δ::Tuple{Vector{Float6
4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0

julia> using CUDA

julia> CUDA.allowscalar(false)

julia> Zygote.hessian(x -> sum(x.a), p |> cu)
ERROR: Scalar indexing is disallowed.
Invocation of getindex resulted in scalar indexing of a GPU array.
This is typically caused by calling an iterating implementation of a method.
Such implementations *do not* execute on the GPU, but very slowly on the CPU,
and therefore are only permitted from the REPL for prototyping purposes.
If you did intend to index this array, annotate the caller with @allowscalar.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] assertscalar(op::String)
    @ GPUArraysCore ~/.julia/packages/GPUArraysCore/uOYfN/src/GPUArraysCore.jl:103
  [3] getindex
    @ Zygote ~/.julia/packages/GPUArrays/EZkix/src/host/indexing.jl:9 [inlined]
  [4] getindex (repeats 2 times)
    @ Zygote ~/.julia/dev/ComponentArrays/src/array_interface.jl:94 [inlined]
  [5] extract(xs::ComponentVector{ForwardDiff.Dual{Nothing, Float32, 4}, ComponentVector{ForwardDiff.Dual{…}, CuArray{…}, Tuple{…}}, Tuple{Axis{…}}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:23
  [6] forward_jacobian(f::Zygote.var"#125#126"{var"#41#42"}, x::ComponentVector{Float32, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Axis{…}}}, ::Val{4})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:29
  [7] forward_jacobian(f::Function, x::ComponentVector{Float32, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Axis{(a = 1:2, b = 3:4)}}}; chunk_threshold::Int64)
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:44
  [8] forward_jacobian
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:42 [inlined]
  [9] hessian_dual
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:64 [inlined]
 [10] hessian(f::Function, x::ComponentVector{Float32, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Axis{(a = 1:2, b = 3:4)}}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:62
 [11] top-level scope
    @ REPL[25]:1
 [12] top-level scope
    @ ~/.julia/packages/CUDA/nbRJk/src/initialization.jl:205
Some type information was truncated. Use `show(err)` to see complete types.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the Zygote.hessian, hessian_reverse, and CUDA failures from the issue first. Inspect src/array_interface.jl, src/namedtuple_interface.jl, and src/compat/chainrulescore.jl alongside the cited Zygote stack frames to understand the ComponentVector interface involved. Done means the reported Hessian paths no longer fail for named components and the CUDA example behaves as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.