aws / aws/amazon-sagemaker-examples
[Bug Report] rl_cartpole_batch_coach example gives an error when training
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
[rl_cartpole_batch_coach.ipynb](https://github.com/aws/amazon-sagemaker-examples/blob/master/reinforcement_learning/rl_cartpole_batch_coach/rl_cartpole_batch_coach.ipynb)
**Describe the bug**
I am running the RL cartpole batch learning example notebook and I get an error when running `estimator.fit()`. It appears to complete all 30 training epochs, but then encounters an error. I am doing this in local model using the `conda_mxnet_p36` environment, but I get a similar error in the same place when not using local mode. I tried a couple other environments and got the same errors.
Here are the last few lines of output:
```
02hifyb2ec-algo-1-eit01 | Training Batch RL Models> Epoch=27, Reward Model Loss=9.223778806334698e-07
02hifyb2ec-algo-1-eit01 | Training Batch RL Models> Epoch=28, Reward Model Loss=8.255876659968953e-07
02hifyb2ec-algo-1-eit01 | Training Batch RL Models> Epoch=29, Reward Model Loss=7.376795045304849e-07
02hifyb2ec-algo-1-eit01 | 2021-05-07 15:27:07,541 sagemaker-containers ERROR ExecuteUserScriptError:
02hifyb2ec-algo-1-eit01 | Command "/usr/bin/python train-coach.py --RLCOACH_PRESET preset-cartpole-ddqnbcq --save_model 1"
02hifyb2ec-algo-1-eit01 exited with code 1
1
Aborting on container exit...
```
And here is the traceback:
```
Failed to delete: /tmp/tmpk8k59j5a/algo-1-eit01 Please remove it manually.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/local/image.py in train(self, input_data_config, output_data_config, hyperparameters, job_name)
237 try:
--> 238 _stream_output(process)
239 except RuntimeError as e:
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/local/image.py in _stream_output(process)
893 if exit_code != 0:
--> 894 raise RuntimeError("Process exited with code: %s" % exit_code)
895
RuntimeError: Process exited with code: 1
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
in
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config)
667 self._prepare_for_training(job_name=job_name)
668
--> 669 self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config)
670 self.jobs.append(self.latest_training_job)
671 if wait:
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/estimator.py in start_new(cls, estimator, inputs, experiment_config)
1430 """
1431 train_args = cls._get_train_args(estimator, inputs, experiment_config)
-> 1432 estimator.sagemaker_session.train(**train_args)
1433
1434 return cls(estimator.sagemaker_session, estimator._current_job_name)
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/session.py in train(self, input_mode, input_config, role, job_name, output_config, resource_config, vpc_config, hyperparameters, stop_condition, tags, metric_definitions, enable_network_isolation, image_uri, algorithm_arn, encrypt_inter_container_traffic, use_spot_instances, checkpoint_s3_uri, checkpoint_local_path, experiment_config, debugger_rule_configs, debugger_hook_config, tensorboard_output_config, enable_sagemaker_metrics, profiler_rule_configs, profiler_config, environment)
565 LOGGER.info("Creating training-job with name: %s", job_name)
566 LOGGER.debug("train request: %s", json.dumps(train_request, indent=4))
--> 567 self.sagemaker_client.create_training_job(**train_request)
568
569 def _get_train_request( # noqa: C901
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/local/local_session.py in create_training_job(self, TrainingJobName, AlgorithmSpecification, OutputDataConfig, ResourceConfig, InputDataConfig, **kwargs)
184 hyperparameters = kwargs["HyperParameters"] if "HyperParameters" in kwargs else {}
185 logger.info("Starting training job")
--> 186 training_job.start(InputDataConfig, OutputDataConfig, hyperparameters, TrainingJobName)
187
188 LocalSagemakerClient._training_jobs[TrainingJobName] = training_job
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/local/entities.py in start(self, input_data_config, output_data_config, hyperparameters, job_name)
219
220 self.model_artifacts = self.container.train(
--> 221 input_data_config, output_data_config, hyperparameters, job_name
222 )
223 self.end_time = datetime.datetime.now()
~/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/sagemaker/local/image.py in train(self, input_data_config, output_data_config, hyperparameters, job_name)
241 # which contains the exit code and append the command line to it.
242 msg = "Failed to run: %s, %s" % (compose_command, str(e))
--> 243 raise RuntimeError(msg)
244 finally:
245 artifacts = self.retrieve_artifacts(compose_data, output_data_config, job_name)
RuntimeError: Failed to run: ['docker-compose', '-f', '/tmp/tmpk8k59j5a/docker-compose.yaml', 'up', '--build', '--abort-on-container-exit'], Process exited with code: 1
```
**To Reproduce**
- Clone this repo into a Sagemaker notebook instance.
- Open `rl_cartpole_batch_coach.ipynb` in JupyterLab.
- In the the 4th code cell, change `local_mode = Flase` to `local_mode = True` (but I get a similar error when running not in local mode).
- Run the code cells in order. All run successfully until the 14th cell, which produces the error.
Contributor guide
Research direction
Open reinforcement_learning/rl_cartpole_batch_coach/rl_cartpole_batch_coach.ipynb and reproduce the failure from the 14th cell, first checking the training output around the final epoch and the train-coach.py command. Compare local and non-local runs, then verify that estimator.fit() completes successfully without the container exit error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100