Some thoughts on Wirtinger

This comes up in discussion on Zygote
https://github.com/FluxML/Zygote.jl/issues/142
https://github.com/FluxML/Zygote.jl/pull/291

Mike/Zygote doesn't like Wirtinger.
Which I personally can get behind, since it is well outside the math I use.
And anything to do with them is a maintaince issue because
I think of the people the contribute to this repo only @simeonschaub and @jrevels properly get them,
(both in terms of the how, and the why)

Zygote never wants to see a Wirtinger differential result.
I think though that might mean also that Zygote wants to treat nonholomorphic functions as nondifferentiable?

Would it make sense to define

function extern(w::Wirtinger)
    iszero(extern(wirtinger_conjugate(w))) || throw(DomainError("Not holomorphic"))
    return extern(wirtinger_primal(w))
end 

Or the equiv inside Zygote.

Could we have a seperate rrule/frule
for Wirtinger that is allowed to return Wirtinger
and normal use of rrule and frule don't.
Then general_rrule and general_frule would be fallback to rrule and frule if there is no special overload for the Wirtinger case defined?

Start with the linked Zygote issue and pull request, then read the named Wirtinger, extern, rrule, frule, general_rrule, and general_frule entry points. This issue needs an agreed policy for Wirtinger results and a defined API before implementation can be considered done.