Manually inject derivative information that can't be computed by AD

I have a function that is computed in a "complicated enough" way for Automatic Differentiation to not be suitable for it. I can however get the derivative in some other way.

I want to use this function as part of another function where the rest can be AD:ed so I need a way to overload the ForwardDiff call to this function and inject the returned derivative.

I have started a bit but I know I am doing it wrong because I don't know how to propagate correctly my computed derivative to the Partials correctly.

using ForwardDiff

# Function that in reality can't be differentiated with AD
function ff(x)
    @assert length(x) == 3
    y = 2*x
    return y
end

# Derivative to above function
function ff_diff(x)
    @assert length(x) == 3
    return 2*eye(3)
end

# Function that I want to use AD on that uses f
function g(x)
    a = 2*x
    b = ff(a)
    return 5*b
end

typealias GradNumFloat3 ForwardDiff.GradientNumber{3,Float64,Tuple{Float64,Float64,Float64}}
typealias PartialFloat3 ForwardDiff.Partials{Float64,Tuple{Float64,Float64,Float64}}

#Overload f
function ff(x::Vector{GradNumFloat3})
    println("I got called")
    # Extract the values
    x_val = ForwardDiff.get_value(x)
    # Call f and get the derivative
    y = ff(x_val)
    dydx = ff_diff(x_val)
    # Insert back 
    # THIS IS WRONG because it ignores the already stored paritals in x
    nums = [GradNumFloat3(y[i], PartialFloat3((dydx[:, i]...))) for i in 1:length(x)]
    return nums
end

ForwardDiff.jacobian(g, rand(3))

# I got called
# 3x3 Array{Float64,2}:
#  10.0   0.0   0.0
#  0.0  10.0   0.0
#  0.0   0.0  10.0

# Above should be 20.0 on diagonal

Any tips on how to do this correctly? Maybe we could put it in the documentation if there is a good way to currently do it.

Start with the ForwardDiff overload and the Partials and GradientNumber types shown in the issue, comparing the custom ff method with how input partials are propagated. Run ForwardDiff.jacobian(g, rand(3)) and verify that the resulting diagonal is 20.0, confirming that the injected derivative composes with the surrounding differentiation.