Order of columns and model accuracy (AUC, F1) on XGBRegressor
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
**Context**
I have a very large number of input variables (100s), but only a handful (~50) matters.
I use XGBRegressor to build classification models (xgboost 0.90)
The model's accuracy is quite good (AUC 0.875, F1 0.846)
**Problem**
1. Rebuilding the model using only the important features gives a lower accuracy (AUC 0.872, F1 0.750) than the original model (getting the list of 39 features using `my_model_xgboost.get_booster().get_score`)
2. Rebuilding the model using Shap's most important features gives the same accuracy as the original model. Shap outputs the same 39 columns as above, but in a different order
This does not reproduce on tiny datasets, like the Iris one.
Given that the drop in accuracy is quite significant, I would consider this a bug in the optimisation step of XGBoost. I understand that with equal importance XGBoost would pick the first one, but this does not seem like an equal importance step.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.