ENH: GMM with instrument callback, optimal instruments

Just an idea while writing a subclass of GMM:

optimal instruments may/will depend on the estimated parameters. Currently, the instruments are always fixed.

One example is nonlinear least squares replication by GMM where the instruments are the Jacobian of the function (or something like this).

• iterative optimization: update instruments together with a GMM.fit loop. I doubt it's computationally efficient to iterate gmm.fit (inner loop) until convergence for each values of the instruments (outer loop)
• joint optimization: using instrument as a function or callback. In this case we will need to separate optimization from inference. AFAIR, the instruments are treated as fixed when we calculate weight and covariance matrix (derivative of moment conditions does not include derivative of instruments which are assumed fixed)

context is roughly (what I remember) (with special reference to treatment effects)

We want to use weights for the observations in the moment conditions. With two stage estimation we can treat "nuisance" parameters as fixed (like variance, heteroscedasticity in LEF models), or we can estimate those parameters jointly in one big GMM problem.
For example GEE treats covariance parameters as fixed for the GMM/GEE inference.

In treatment effects with inverse propensity score weighting, we can treat the weights as fixed or include the estimating equations of the treatment selection model in GMM.

The theory roughly AFAIU (might not apply to all models within the treatment effects category)
With parametric LEF (QMLE) models the variance or weight parameters are separable and we can treat them as fixed (and use sandwich cov_params). However, assumption in treatment effects literature is semiparametric, the mean function can be misspecified in double robust estimators. Additionally, taking effect of first stage estimators into account might improve small sample properties.
(Stata calculates standard errors based on joint moment conditions. Some references mention that we need to correct for first stage estimation. Topel and similar, which can be done automatically in joint GMM moment conditions.)

Start by reading the existing GMM implementation and the GMM.fit loop, then trace how instruments, optimization, weights, and covariance inference are handled. The issue describes callback-based or joint treatment of parameter-dependent instruments, so the completed work would need a concrete design that separates optimization from inference and defines how first-stage effects are incorporated.