aws / aws/amazon-sagemaker-examples

script-mode-container-2.ipynb failed CI

Open
#2,518 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Link to the notebook:
https://github.com/aws/amazon-sagemaker-examples/blob/master/advanced_functionality/custom-training-containers/script-mode-container-2/notebook/script-mode-container-2.ipynb

Error:

---------------------------------------------------------------------------
Exception encountered at "In [10]":
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
in
23 val_config = sagemaker.session.s3_input('s3://{0}/{1}/val/'.format(bucket, prefix), content_type='text/csv')
24
---> 25 est.fit({'train': train_config, 'validation': val_config })

/opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)
667 self._prepare_for_training(job_name=job_name)
668
--> 669 self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config)
670 self.jobs.append(self.latest_training_job)
671

[...]

Exception: Failed to run docker,images,-q,521695447989.dkr.ecr.us-west-2.amazonaws.com/sagemaker-training-containers/script-mode-container-2:latest

Contributor guide

Open the contributing guide

Research direction

Open advanced_functionality/custom-training-containers/script-mode-container-2/notebook/script-mode-container-2.ipynb and inspect In [10], where est.fit fails while running the Docker command. Reproduce the CI failure and investigate the container execution environment; done means the notebook completes that training step successfully in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.