svd_rev issue with orthogonal matricies

using Zygote
using LinearAlgebra

r = rand(8,8); Σ = r' * r
foo(X) = tr(svd(X).U)
_orthogonal(X) = svd(X).U * svd(X).V'

Zygote.gradient(foo, Σ) # Works
Zygote.gradient(foo, 1.0 * Matrix(I(5))) # Nan
Zygote.gradient(foo, _orthogonal(rand(5,7))) # NaN/Inf

This reads as an edge case, but the reason this came up because of interest in introducing an orthogonality constraint in a pipeline. As the matrix is constrainted to be orthogonal, a natural initialisation would be also orthogonal, however it leads to the above issue (this essentially does the same thing):

using ParameterHandling: value, orthogonal

bar(X) = tr(value(orthogonal(X)))
Zygote.gradient(bar, Σ) # Works

# A natural initialisation for an orthogonal constrained matrix is orthogonal
Zygote.gradient(bar, _orthogonal(Σ)) # Fails

IIUC, I believe that the for orthogonal matrix as the singular values are 1 the F in the svd rule will explode.

A workaround was to add a small noise term suggesting that somewhere perhaps this should cancel? - the UᵀŪ - ŪᵀU) and (VᵀV̄ - V̄ᵀV terms) but idk

Zygote.gradient(bar, _orthogonal(Σ) + 1e-10 * Diagonal(rand(size(Σ, 1)))) # Works
# ([3.032964106070741e-29 -6.862457650747e-16 … -8.654203384732967e-16 2.8416261314487268e-15; 6.862728701290122e-16 -2.4424906535614213e-15 … 1.1744348982906105e-15 -1.2198799099766983e-15; … ; 8.654203384732967e-16 -1.4343655928804322e-15 … -1.7763568394002505e-15 2.310651670001107e-15; -2.841639683975883e-15 1.0009591625611408e-15 … -1.7551064768195346e-15 -1.5543122344752192e-15],)

There doesn't seem to be anything wrong with the rule themselves, and the workaround is reasonable once it's understood what's occuring, however as this started impacting another internally I thought i'd raise this as an issue. If the diagnosis is correct, are there any approaches to make this nice in the scenario described?

Start with the SVD rule in src/rulesets/LinearAlgebra/factorization.jl at the linked location, then reproduce the three Zygote.gradient examples involving identity and orthogonal matrices. Investigate the reported singular-value degeneracy and determine the expected behavior for orthogonal inputs. Done requires an agreed approach and a regression test or documented limitation, but the issue does not specify which outcome maintainers prefer.