aws / aws/amazon-sagemaker-examples

tensorflow2_smdataparallel_bert_demo.ipynb failed CI

Open
#2,259 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Link to the notebook:
https://github.com/aws/amazon-sagemaker-examples/blob/master/training/distributed_training/tensorflow/data_parallel/bert/tensorflow2_smdataparallel_bert_demo.ipynb

Error:

---------------------------------------------------------------------------
Exception encountered at "In [12]":
---------------------------------------------------------------------------
ParamValidationError Traceback (most recent call last)
in
1 # Submit SageMaker training job
----> 2 estimator.fit(inputs=data_channels, job_name=job_name)

/usr/local/lib/python3.7/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)
667 self._prepare_for_training(job_name=job_name)
668
--> 669 self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config)
670 self.jobs.append(self.latest_training_job)
671 if wait:

/usr/local/lib/python3.7/site-packages/sagemaker/estimator.py in start_new(cls, estimator, inputs,

[...]

Invalid length for parameter InputDataConfig[0].DataSource.FileSystemDataSource.FileSystemId, value: 8, valid min length: 11

Contributor guide

Open the contributing guide

Research direction

Start with training/distributed_training/tensorflow/data_parallel/bert/tensorflow2_smdataparallel_bert_demo.ipynb and inspect cell In [12], where estimator.fit(inputs=data_channels, job_name=job_name) submits the SageMaker training job. Reproduce the failure and investigate the Invalid length for InputDataConfig[0].DataSource.FileSystemDataSource.FileSystemId error; done means the notebook completes CI without this validation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python, tensorflow
Domain
cloud, distributed-systems, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.