aws / aws/amazon-sagemaker-examples

[Bug Report] Sagemaker Model Monitor Visualization with BatchTransform

Open
#3,915 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/tree/1185672adb2e68eed76fd7b7a2bf16b9ab418d97/sagemaker_model_monitor/visualization

**Describe the bug**
I am using ModelMonitor with BatchTransform functionality. When I try to visualize the violations, the above notebook is not working.

**To reproduce**
Run MonitorBatchTransformStep and then evaluate the report using above notebook.
Then the error bellow appears. The main reason is that there is no inferred feature.

**Logs**
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
in
6 baseline_statistics=baseline_statistics,
7 latest_statistics=execution_statistics,
----> 8 violations=violation.body_dict["violations"],
9 )

~/chat-propensity/aws-chat-propensity-deploy-p-lceflgsg0cqh/pipelines/project/utils.py in show_violation_df(baseline_statistics, latest_statistics, violations)
211 def show_violation_df(baseline_statistics, latest_statistics, violations):
212 violation_df, violation_df_style = create_violation_df(
--> 213 baseline_statistics, latest_statistics, violations
214 )
215 return style_violation_df(violation_df, violation_df_style)

~/chat-propensity/aws-chat-propensity-deploy-p-lceflgsg0cqh/pipelines/project/utils.py in create_violation_df(baseline_statistics, latest_statistics, violations)
151 feature_names = list(features.keys())
152 feature_names.sort()
--> 153 data_type_df, data_type_df_style = create_data_type_df(feature_names, features)
154 completeness_df, completeness_df_style = create_completeness_df(feature_names, features)
155 baseline_drift_df, baseline_drift_df_style = create_baseline_drift_df(feature_names, features)

~/chat-propensity/aws-chat-propensity-deploy-p-lceflgsg0cqh/pipelines/project/utils.py in create_data_type_df(feature_names, features)
56 for feature_name in feature_names:
57 feature = features[feature_name]
---> 58 latest = feature["latest"]["inferred_type"]
59 violation = violation_exists(feature, "data_type_check")
60 rows.append([latest])

KeyError: 'latest'

Contributor guide

Open the contributing guide

Research direction

Start with the visualization notebook at the linked commit, then run MonitorBatchTransformStep and evaluate its report as described. Trace the failure around the missing inferred feature and the KeyError for "latest"; done means the violations visualization works for this batch-transform report.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python
Domain
data-visualization, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.