Misguided Implementation of Saabas' method in `RegTree::CalculateContributions(Approx)`
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
Despite citing the original blog post [Interpreting random forests](https://blog.datadive.net/interpreting-random-forests/), the current implementation of Saabas' method in `RegTree::CalculateContributionsApprox` seems misguided.
https://github.com/dmlc/xgboost/blob/7d43e74e71c1a0f6526fda78d46395c883d14cb0/src/tree/tree_model.cc#L1214-L1218
In particular, the `mean_values` parameter is filled by the function `FillNodeMeanValues`, which as its name suggests, simply calculates the average response within a particular node.
https://github.com/dmlc/xgboost/blob/7d43e74e71c1a0f6526fda78d46395c883d14cb0/src/predictor/cpu_predictor.cc#L260-L275
However, I argue that we should remove the parameter `mean_values` and use `(*this)[nid].LeafValue()` instead. This is because the idea of Saabas' method is to see how the prediction changes as we go deeper into the tree.
To quote the blog post by Saabas,
> What's novel here is that you can see the breakdown of the prediction, written down in terms of value changes along the prediction path, together with feature names that “caused” every value change due to being in the guard (the numbers are approximate due to rounding).
For random forests, the prediction/weight coincides with the average value within that node, i.e. $-\frac{G_j}{H_j}$, which is what the current implementation calculates. However, for XGBoost with regularization, the value should be $-\frac{\operatorname{ThresholdL1}(G_j-\alpha)}{H_j+\lambda}$, where $\alpha$ and $\lambda$ are L1 and L2 regularization parameters respectively.
https://github.com/dmlc/xgboost/blob/7d43e74e71c1a0f6526fda78d46395c883d14cb0/src/tree/param.h#L246-L258
Additionally, we are using Saabas' method as an approximation for (Tree)SHAP in this context. The SHAP value for a feature is the average change in **model output** by conditioning on that feature when introducing features one at a time over all feature orderings, and Saabas' approximates it by only considering a particular ordering of the features, i.e. the one specified by the tree. This would not be the case if we calculate the average response instead of the actual model prediction.
To sum up, I believe the `RegTree::CalculateContributionsApprox` method should use the weight/prediction value in each node, instead of `mean_values`. I'm more than happy to open a PR to address that. Please let me know what you think.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.