aws / aws/amazon-sagemaker-examples

tf2-keras-default-container.ipynb failed CI

Open
#2,354 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Link to the notebook:
https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker-debugger/tensorflow2/tensorflow2_zero_code_change/tf2-keras-default-container.ipynb

Error:

---------------------------------------------------------------------------
Exception encountered at "In [10]":
---------------------------------------------------------------------------
MissingCollectionFiles Traceback (most recent call last)
in
1 from smdebug.trials import create_trial
----> 2 trial = create_trial(estimator.latest_job_debugger_artifacts_path())

/usr/local/lib/python3.7/site-packages/smdebug/trials/utils.py in create_trial(path, name, profiler, output_dir, **kwargs)
21 return ProfilerTrial(name=name, trial_dir=path, output_dir=output_dir, **kwargs)
22 if s3:
---> 23 return S3Trial(name=name, bucket_name=bucket_name, prefix_name=prefix_name, **kwargs)
24 else:
25 return LocalTrial(name=name, dirname=path, **kwargs)

/usr/local/lib/python3.7/site-packages/smdebug/trials/s3_t

[...]

MissingCollectionFiles: Training job has ended. All the collection files could not be loaded

Contributor guide

Open the contributing guide

Research direction

Open sagemaker-debugger/tensorflow2/tensorflow2_zero_code_change/tf2-keras-default-container.ipynb and reproduce the CI failure at In [10], where create_trial loads the debugger artifacts. Inspect the MissingCollectionFiles error and the notebook's training-job setup. Done means the notebook completes without this exception in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python, tensorflow
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.