aws / aws/amazon-sagemaker-examples

callback_bottleneck.ipynb failed CI

Open
#2,360 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Link to the notebook:
https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker-debugger/tensorflow_profiling/callback_bottleneck.ipynb

Error:

---------------------------------------------------------------------------
Exception encountered at "In [11]":
---------------------------------------------------------------------------
ClientError Traceback (most recent call last)
in
5 region = 'us-east-1'
6
----> 7 tj = TrainingJob(training_job_name, region)
8
9 pf = PandasFrame(tj.profiler_s3_output_path)

/usr/local/lib/python3.7/site-packages/smdebug/profiler/analysis/notebook_utils/training_job.py in __init__(self, training_job_name, region)
22 self.sm_client = boto3.client("sagemaker", region_name=region)
23 self.profiler_config, self.profiler_s3_output_path = (
---> 24 self.get_config_and_profiler_s3_output_path()
25 )
26 self.system_metrics_reader

[...]

ClientError: An error occurred (ValidationException) when calling the DescribeTrainingJob operation: Requested resource not found.

Contributor guide

Open the contributing guide

Research direction

Open sagemaker-debugger/tensorflow_profiling/callback_bottleneck.ipynb and inspect In [11], starting with TrainingJob(training_job_name, region) and the reported DescribeTrainingJob error. Reproduce the CI execution and determine why the requested training job is unavailable; done means the notebook completes that step without the resource-not-found failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.