dmlc / dmlc/xgboost

References / explanations for contribution predictions

Open
#12,241 12 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

XGBoost offers an option `pred_contribs`, for which the docs explain to some degree what the result is supposed to contain, but they do not explain how those are computed, and do not cite any reference.

If I understand it correctly, those contributions are based on SHAP, which comes from [this paper](https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf):
> Lundberg, Scott M., and Su-In Lee.
> "A unified approach to interpreting model predictions."
> Advances in neural information processing systems 30 (2017).

Is that the case?

If I understand the paper correctly, when it computes the effect of a feature being excluded, it is supposed to take a weighted average of the predictions that would be obtained under the different possible values, but it appears XGBoost is using the sum of hessians, which would only correspond to a proportional split of the data for squared loss:
https://github.com/dmlc/xgboost/blob/f41697435c9b60c84b226304401e3183207fea73/src/predictor/interpretability/shap.cc#L57

Where can I find some reference describing the methodology used here and these kinds of choices?

Contributor guide

No contributing guide indexed for this repository

Research direction

Read the existing pred_contribs documentation, the cited SHAP paper, and src/predictor/interpretability/shap.cc around line 57. Determine how XGBoost computes excluded-feature contributions and whether the hessian weighting matches the paper; done means documenting the methodology, relevant choices, and authoritative references.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.