awslabs / awslabs/sagemaker-debugger
Full shap values
- Dominant language
- Python
- Stars
- 165
- Forks
- 82
- PR merge metrics
- No merged PRs in 30d
Description
When retrieving `full_shap` values, debugger returns a matrix in the shape of `number of training samples, number of features` e.g.
```
for index,i in enumerate(trial.tensor_names(regex='full_shap')):
tensor = trial.tensor(i).value(step_num=50)
print(i, tensor.shape)
full_shap/bias (26048, 13)
full_shap/f0 (26048, 13)
full_shap/f1 (26048, 13)
full_shap/f10 (26048, 13)
full_shap/f11 (26048, 13)
full_shap/f2 (26048, 13)
full_shap/f3 (26048, 13)
full_shap/f4 (26048, 13)
full_shap/f5 (26048, 13)
full_shap/f6 (26048, 13)
full_shap/f7 (26048, 13)
full_shap/f8 (26048, 13)
full_shap/f9 (26048, 13)
```
Expected shape of the tensor would be `(26048,1)` The issue happens in this [line](https://github.com/awslabs/sagemaker-debugger/blob/master/smdebug/xgboost/hook.py#L270-L271):
```
for feature_id, feature_name in enumerate(feature_names):
self._save_for_tensor(f"full_shap/{feature_name}", self._full_shap_values)
```
The following should fix it:
```
for feature_id, feature_name in enumerate(feature_names):
self._save_for_tensor(f"full_shap/{feature_name}", self._full_shap_values[:, feature_id])
```
Contributor guide
Research direction
Start in smdebug/xgboost/hook.py at lines 270-271, where full_shap tensors are saved, and inspect how feature names map to the SHAP values. Verify the change against the reported example; done means each full_shap feature tensor has the expected (26048, 1) shape rather than (26048, 13).
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100