awslabs / awslabs/sagemaker-debugger

Full shap values

Open
#294 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
165
Forks
82
PR merge metrics
No merged PRs in 30d

Description

When retrieving `full_shap` values, debugger returns a matrix in the shape of `number of training samples, number of features` e.g.
```
for index,i in enumerate(trial.tensor_names(regex='full_shap')):
tensor = trial.tensor(i).value(step_num=50)
print(i, tensor.shape)

full_shap/bias (26048, 13)
full_shap/f0 (26048, 13)
full_shap/f1 (26048, 13)
full_shap/f10 (26048, 13)
full_shap/f11 (26048, 13)
full_shap/f2 (26048, 13)
full_shap/f3 (26048, 13)
full_shap/f4 (26048, 13)
full_shap/f5 (26048, 13)
full_shap/f6 (26048, 13)
full_shap/f7 (26048, 13)
full_shap/f8 (26048, 13)
full_shap/f9 (26048, 13)
```
Expected shape of the tensor would be `(26048,1)` The issue happens in this [line](https://github.com/awslabs/sagemaker-debugger/blob/master/smdebug/xgboost/hook.py#L270-L271):
```
for feature_id, feature_name in enumerate(feature_names):
self._save_for_tensor(f"full_shap/{feature_name}", self._full_shap_values)
```
The following should fix it:
```
for feature_id, feature_name in enumerate(feature_names):
self._save_for_tensor(f"full_shap/{feature_name}", self._full_shap_values[:, feature_id])
```

Contributor guide

Open the contributing guide

Research direction

Start in smdebug/xgboost/hook.py at lines 270-271, where full_shap tensors are saved, and inspect how feature names map to the SHAP values. Verify the change against the reported example; done means each full_shap feature tensor has the expected (26048, 1) shape rather than (26048, 13).

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.