aws / aws/amazon-sagemaker-examples

[Bug Report] trial.collection('feature_importance") not giving all values

Open
#3,645 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-debugger/xgboost_census_explanations/xgboost-census-debugger-rules.ipynb

**Describe the bug**
The dataset for this Notebook has 11 features, however, `plot_feature_importance(trial, importance_type="cover")` only shows 6 of them. (Same if using another *importance_type*)
I found that this happens because, in `plot_collection(...)` the call to `trial.collection('feature_importance")` only returns these 6 values.
I've changed, in `plot_collection(...)` the line:
`tensors = trial.collection(collection_name).tensor_names`
for:
`tensors = trial.tensor_names()`
And it works for *feature_importance*, but this gives error for *metrics* and *average_shap* collections.
Why does `trial.collection('feature_importance")` doesn't return all values for all features?

**To reproduce**
Simply execute the Notebook.
`plot_feature_importance(trial, importance_type="cover")` plots 6 lines, when there's 11 features in the dataset

Contributor guide

Open the contributing guide

Research direction

Run the linked xgboost-census-debugger-rules.ipynb and reproduce the six-line output from plot_feature_importance(trial, importance_type="cover"). Inspect plot_collection(...), especially trial.collection(collection_name).tensor_names, and compare it with trial.tensor_names(); done means all 11 feature values appear without breaking metrics or average_shap collections.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
data-visualization, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.