Change forcing sequence in newton-cg solver of LogisticRegression

Describe the workflow you want to enable

I'd like to have faster convergence of the "newton-cg" solver of LogisticRegression based on scientific publications with empirical studies as done in A Study on Truncated Newton Methods for Linear Classification (2022) (free pdf version).

Describe your proposed solution

It is about the inner stopping criterion in a truncated Newton solver, i.e. when should the inner solver for "hessian @ coefficients = -gradient" stop.

$eta = \eta$ is the forcing sequence.

Current stopping criterion

$residual ratio = \frac{\rVert res\lVert_1}{\rVert grad \lVert_1} \leq \eta$ with $res = residual = grad - hess @ coef$ and $\eta = \min([0.5, \sqrt{\rVert grad \lVert_1]})$ (this eta is called adaptive forcing sequence.

Proposed stopping criterion

As recommended by Chapter VII.

• Replace residual ratio with the quadratic approximation ratio $j\frac{Q_j - Q_{j-1}}{Q_j}$ and $Q_j = grad @ coef_j + \frac{1}{2} coef_j^T @ hessian @ coef_j$ and $j$ is the inner iteration number.
• Optionally replace L1-norm by L2-norm. For the quadratic ratio, this does not matter much.

Describe alternatives you've considered, if relevant

No response

Additional context

No response

Start at the LogisticRegression implementation of the "newton-cg" solver and trace the inner stopping criterion for the Hessian-and-gradient solve. Compare the current adaptive forcing sequence with the quadratic approximation ratio in the cited paper, then identify or add solver tests that establish convergence behavior and the selected norm before considering the change complete.