aws / aws/amazon-sagemaker-examples

SageMaker-Model-Monitor-tensorflow.ipynb failed CI

Open
#2,342 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Link to the notebook:
https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker_model_monitor/tensorflow/SageMaker-Model-Monitor-tensorflow.ipynb

Error:

---------------------------------------------------------------------------
Exception encountered at "In [34]":
---------------------------------------------------------------------------
ClientError Traceback (most recent call last)
in
----> 1 my_default_monitor.delete_monitoring_schedule()
2 time.sleep(120) # actually wait for the deletion

/usr/local/lib/python3.7/site-packages/sagemaker/model_monitor/model_monitoring.py in delete_monitoring_schedule(self)
1945 def delete_monitoring_schedule(self):
1946 """Deletes the monitoring schedule and its job definition."""
-> 1947 super(DefaultModelMonitor, self).delete_monitoring_schedule()
1948 if self.job_definition_name is not None:
1949 # Delete job definition.

/usr/local/lib/pytho

[...]

ClientError: An error occurred (ValidationException) when calling the DeleteMonitoringSchedule operation: Monitoring schedule in status Pending

Contributor guide

Open the contributing guide

Research direction

Start with sagemaker_model_monitor/tensorflow/SageMaker-Model-Monitor-tensorflow.ipynb and reproduce the failure at In [34], where my_default_monitor.delete_monitoring_schedule() is called. Investigate the reported ValidationException for a schedule in Pending status; done means the notebook's CI run completes without this error and monitoring resources are cleaned up.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python, tensorflow
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.