aws / aws/amazon-sagemaker-examples

Preprocessing script logs sagemaker model monitoring.

Open
#1,134 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

While creating a monitoring job for an endpoint I attached a preprocessing script to flatten the captured data but my script fails with an invocation error. Furthermore, I am unable to see any logs from the preprocessing script in the cloud watch just an innovation error in monitoring job logs.
Tried writing into the '**/opt/ml/output/message**' file after encoding the logs in UTF-8 but could not see any logs from the script in monitoring job logs.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the SageMaker model monitoring job with the attached preprocessing script and inspect the monitoring job logs in CloudWatch. Check whether writing UTF-8 encoded output to /opt/ml/output/message is supported and whether the invocation error identifies the script failure. Done means the preprocessing script runs successfully and its diagnostic logs are visible in the documented monitoring logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, machine-learning, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.