dmlc / dmlc/xgboost

Only analyze features included in model (predcontrib,predinteraction)

Open
#4,998 1 comment 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

This concerns Tree shap via XGBoost in my case using R. Think this is relevant besides R though.

Short:
In R: Matrix/array returned from predict() with predcontrib=TRUE or predinteractions=TRUE is very sparse due to variables not included in model are included in matrix/array. Give user opportunity to only return variables included in model?
In addition, considering returning objects which take less memory than matrix and array?

Long:
Having a large data set (in my case 350k x 450k), an ensemble method such as XGBoost may include after training only a small subset of features in the trees. At least when using R, using predict.xgb.Booster() with either predcontrib = TRUE or predinteractions = TRUE will return a matrix/array with all features from original training data (thereby consisting of a huge amount of zeros), but I only want to analyze those features included in the trees.

I have looked at the source code (XGBoosterPredict_R -> XGBoosterPredict -> Predict -> PredictContribution/PredictInteractionContributions), but unfortunately extremely unexperienced with C++, and so not sure what should be done. Is there a quick fix? If someone would want to help me with this, I would deeply appreciate it. Simply setting model$feature_names = "FeaturesOfInterest" and making sure newdata/validation data only consists of "FeaturesOfInterest" in the R-function predict() does not work, where model is a xgb.Booster-object returned after training. It returns something very strange. I do think something needs to be done in source code.

In addition, as far as I have understood, the array when computing interactions may be very sparse. Maybe consider to return something more memory-efficient than matrix()?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.