Comparing`fit!` and `evaluate!` for `LogisticClassifier` with scikit-learn

This issue is closely related to https://github.com/alan-turing-institute/MLJLinearModels.jl/issues/14

I'm trying to compare performance of fit! and evaluate! in Julia and analogous methods in scikit-learn in Python. The code and data are available by the link: https://gist.github.com/irublev/6e8f928ef78993922aa4b5d735bf3efc

What concerns Julia code, I run it in Windows 10, Julia version 1.6.1 (2021-04-23), MLJ v0.16.4, MLJBase v0.18.6, MLJLinearModels v0.5.4 (and the Python code was run in Python 3.8.6, scikit-learn 0.23.2, numpy 1.19.2).

And the results are as follows:

It should be noted that the corresponding Julia code marked with @time (see https://gist.github.com/irublev/6e8f928ef78993922aa4b5d735bf3efc) was executed several times to exclude compilation time.

And the question I'd like to answer is why MLJ is 2.5-3 times slower than scikit-learn. I tried to make parameters of models in Julia and in Python as close as possible.

But I have not found the way to configure solvers. In Python we have the following list of parameters:

classifier.get_params()
Out[82]: 
{'C': 0.1,
 'class_weight': None,
 'dual': False,
 'fit_intercept': True,
 'intercept_scaling': 1,
 'l1_ratio': None,
 'max_iter': 100,
 'multi_class': 'auto',
 'n_jobs': None,
 'penalty': 'l2',
 'random_state': None,
 'solver': 'lbfgs',
 'tol': 0.0001,

Thus, we have here maximal number of iterations, tolerance, etc. And I'm not sure the comparison above is correct, because I do not know what are the values of the respective parameters for LFBGS solver in Julia.

Could you please say how to make sure that all is configured in Julia exactly as it is in Python? And what may be the reason for such performance of Julia? May it be improved somehow? It does not looke as expect on par or better as pointed in https://github.com/alan-turing-institute/MLJLinearModels.jl/issues/14

Start with the linked Gist and reproduce the Julia and Python timing comparison using the versions listed in the issue. Read related issue #14 and check how LogisticClassifier exposes solver, tolerance, and iteration settings. Done means documenting a like-for-like comparison or identifying a specific performance problem and its scope.