RFC Expose a plublic method to compute the objective function

I think that it would be valuable that all estimators that optimize some objective function expose a public method to compute it.
My main motivation is for the callbacks, for early stopping or monitoring, but I'm sure it would be useful in other contexts.

To really be practical, the signature should be the same across all estimators. What I have in mind is something like:

def objective_function(self, X, y=None, *, sample_weight=None, normalize=False):
    y_pred = self.predict(X, y)
    # or Xt = self.transform(X) because some transformers do optimize an objective function.

    data_fit = <computation of the data fit term>
    penalization = <computation of the penalization term>

    if normalize:                     # allow to return a per sample
        data_fit /= X.shape[0]        # objective function
        penalization /= X.shape[0]    # for convenience
        # X.shape[0] probably needs to be replaced by sample_weight.sum()
 
    return data_fit + penalization, data_fit, penalization

If we want to compute the objective function of the training set during fitting, we could allow to provide the current variables that are required to compute it at a given iteration, encapsulated in a single argument (a dict):

def objective_function(self, X, y=None, *, sample_weight=None, fit_state=None, normalize=False):
    if fit_state is None:
        y_pred = self.predict(X)
    else:
        y_pred = <compute y_pred using the information in fit_state>

    ...

where the content of fit_state would be estimator specific, and detailed in the docstring of objective_function for each estimator.

Start by reviewing the proposed objective_function signatures and how existing scikit-learn estimators use predict, transform, callbacks, and fit state. Compare objective computations across estimators before deciding whether one public API can cover them. Done means the scope and common interface are agreed for the affected estimators; this issue names no files or tests.