RFC make response / inverse link / activation function official

With questions like #29163 and with the private loss functions #15123 (almost everywhere) in place, I would like to discuss to make the inverse link function public.

Models like LogisticRegression or HistGradientBoostingRegressor(loss="poisson") have predictions like inverse_link(raw_prediction(X)) where raw_prediction(X) is the prediction in "link space", e.g. linear predictor ("eta") for linear models.

In line with the most recent nomenclature of HistGradientBoosting*, I propose the following public API for regressors and classifiers:

• raw_predict(X)
• response_function(y_raw) or activation_function(y_raw)
• link_function(y_obs)

Alternatives:

1. estimator.link is a link object which has 2 methods named like above.
2. 1-to-1 with the actual implementation: estimator.loss.link and then as alternative 1. This would also expose the loss function (object), see also #28169.

Further considerations

• This would also make easier/solve #18309
• Does this necessitate a SLEP?

@scikit-learn/communication-team @scikit-learn/contributor-experience-team @scikit-learn/core-devs @scikit-learn/documentation-team ping

Start by reading the linked discussions #29163, #15123, #18309, and #28169, along with the recent HistGradientBoosting nomenclature referenced here. Compare the proposed raw_predict, response_function or activation_function, and link_function APIs with the listed alternatives. Done means the public API direction is agreed, including whether a SLEP is needed.