Analyze the practical relevance of GBT hyperparameters in the accuracy / training speed tradeoff

The following paper ran many hyper-parameter tuning experiments.

• https://arxiv.org/abs/2207.08815
• supporting code: https://github.com/LeoGrin/tabular-benchmark

The raw CSV data is available for download here:

• https://github.com/LeoGrin/tabular-benchmark/tree/main/analyses/results

We could try to mine this data to find hyper-params configuration implemented in xgboost or lightgbm that:

• allow xgboost or lightgbm or catboost to be on the Pareto optimal frontier of the predictive/computational performance tradeoff;
• have no equivalent in scikit-learn.

This would help identify which missing entries of the table in #27873 have the most user-facing impact, and possibly also identify features of xgboost that are not implemented in scikit-learn at all while being very relevant to reach good predictive performance.

Start by reading the linked paper and inspecting the raw CSV files in LeoGrin/tabular-benchmark/analyses/results. Compare the reported XGBoost, LightGBM, and CatBoost configurations with the table in #27873 and scikit-learn equivalents. Done means identifying configurations on the predictive/computational Pareto frontier and documenting which have no meaningful scikit-learn equivalent.