error when trying to futher differentiate using Zygote

Summary: it seems that the finite difference can't be further differentiated by Zygote
Example: If I do

dif(x) = central_fdm(2, 1)(sin, x)

and then call

dif'(0)

I get the following error message:

setindex!(::StaticArrays.SVector{3, Float64}, value, ::Int) is not defined.

error(::String)@error.jl:33
setindex!(::StaticArrays.SVector{3, Float64}, ::Float64, ::Int64)@indexing.jl:3
#637@array.jl:318[inlined]
#2710#back@adjoint.jl:59[inlined]
Pullback@methods.jl:385[inlined]
(::typeof(∂(_estimate_magnitudes)))(::Tuple{Float64, Nothing})@interface2.jl:0
Pullback@methods.jl:365[inlined]
(::typeof(∂(estimate_step)))(::Tuple{Float64, Nothing})@interface2.jl:0
Pullback@methods.jl:193[inlined]
(::typeof(∂(λ)))(::Float64)@interface2.jl:0
Pullback@Other: 1[inlined]
(::typeof(∂(dif)))(::Float64)@interface2.jl:0
(::Zygote.var"#41#42"{typeof(∂(dif))})(::Float64)@interface.jl:41
gradient(::Function, ::Int64)@interface.jl:59
(::Zygote.var"#43#44"{typeof(Main.workspace50.dif)})(::Int64)@interface.jl:62
top-level scope@Local: 1[inlined]

The actual use case is that I'm trying to train a NN where I need to compute the gradient of its parameters w.r.t a loss function that involves finite differences given by central_fdm. Is this possible?

Use the central_fdm and dif example as the entry point, then reproduce the Zygote gradient failure involving StaticArrays.SVector and the setindex! trace. Inspect the pullback path named in adjoint.jl and methods.jl to determine whether further differentiation is supported. Done means establishing a working result or clearly documenting the limitation with a reproducible test case.