aws / aws/amazon-sagemaker-examples
[Bug Report] Sagemaker Model Monitor Visualization with BatchTransform
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/tree/1185672adb2e68eed76fd7b7a2bf16b9ab418d97/sagemaker_model_monitor/visualization
**Describe the bug**
I am using ModelMonitor with BatchTransform functionality. When I try to visualize the violations, the above notebook is not working.
**To reproduce**
Run MonitorBatchTransformStep and then evaluate the report using above notebook.
Then the error bellow appears. The main reason is that there is no inferred feature.
**Logs**
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
in
6 baseline_statistics=baseline_statistics,
7 latest_statistics=execution_statistics,
----> 8 violations=violation.body_dict["violations"],
9 )
~/chat-propensity/aws-chat-propensity-deploy-p-lceflgsg0cqh/pipelines/project/utils.py in show_violation_df(baseline_statistics, latest_statistics, violations)
211 def show_violation_df(baseline_statistics, latest_statistics, violations):
212 violation_df, violation_df_style = create_violation_df(
--> 213 baseline_statistics, latest_statistics, violations
214 )
215 return style_violation_df(violation_df, violation_df_style)
~/chat-propensity/aws-chat-propensity-deploy-p-lceflgsg0cqh/pipelines/project/utils.py in create_violation_df(baseline_statistics, latest_statistics, violations)
151 feature_names = list(features.keys())
152 feature_names.sort()
--> 153 data_type_df, data_type_df_style = create_data_type_df(feature_names, features)
154 completeness_df, completeness_df_style = create_completeness_df(feature_names, features)
155 baseline_drift_df, baseline_drift_df_style = create_baseline_drift_df(feature_names, features)
~/chat-propensity/aws-chat-propensity-deploy-p-lceflgsg0cqh/pipelines/project/utils.py in create_data_type_df(feature_names, features)
56 for feature_name in feature_names:
57 feature = features[feature_name]
---> 58 latest = feature["latest"]["inferred_type"]
59 violation = violation_exists(feature, "data_type_check")
60 rows.append([latest])
KeyError: 'latest'
Contributor guide
Research direction
Start with the visualization notebook at the linked commit, then run MonitorBatchTransformStep and evaluate its report as described. Trace the failure around the missing inferred feature and the KeyError for "latest"; done means the violations visualization works for this batch-transform report.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python
- Domain
- data-visualization, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100