References / explanations for contribution predictions
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
XGBoost offers an option `pred_contribs`, for which the docs explain to some degree what the result is supposed to contain, but they do not explain how those are computed, and do not cite any reference.
If I understand it correctly, those contributions are based on SHAP, which comes from [this paper](https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf):
> Lundberg, Scott M., and Su-In Lee.
> "A unified approach to interpreting model predictions."
> Advances in neural information processing systems 30 (2017).
Is that the case?
If I understand the paper correctly, when it computes the effect of a feature being excluded, it is supposed to take a weighted average of the predictions that would be obtained under the different possible values, but it appears XGBoost is using the sum of hessians, which would only correspond to a proportional split of the data for squared loss:
https://github.com/dmlc/xgboost/blob/f41697435c9b60c84b226304401e3183207fea73/src/predictor/interpretability/shap.cc#L57
Where can I find some reference describing the methodology used here and these kinds of choices?
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the existing pred_contribs documentation, the cited SHAP paper, and src/predictor/interpretability/shap.cc around line 57. Determine how XGBoost computes excluded-feature contributions and whether the hessian weighting matches the paper; done means documenting the methodology, relevant choices, and authoritative references.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100