aws / aws/amazon-sagemaker-examples
Import error in advanced_functionality/autogluon-tabular/AutoGluon_Tabular_SageMaker.ipynb
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Description of the error**
when running the following cell in [AutoGluon_Tabular_SageMaker.ipynb](https://github.com/awslabs/amazon-sagemaker-examples/tree/master/advanced_functionality/autogluon-tabular):
```python
%%time
instance_type = 'ml.m5.2xlarge'
#instance_type = 'local'
ecr_image = f'{ecr_uri_prefix}/{training_algorithm_name}:latest'
estimator = Estimator(image_name=ecr_image,
role=role,
train_instance_count=1,
train_instance_type=instance_type,
hyperparameters=hyperparameters)
estimator.fit(train_s3_path)
```
I got the following error:
```
UnexpectedStatusException: Error for Training job autogluon-sagemaker-training-2020-05-10-07-12-25-701: Failed. Reason: AlgorithmError: ExecuteUserScriptError:
Command "/usr/local/bin/python3.6 train.py --label y"
```
The log provided by training container was bellow.
```
Traceback (most recent call last):
File "train.py", line 20, in
import autogluon as ag
File "package/autogluon/__init__.py", line 10, in
from . import scheduler, searcher, utils
File "package/autogluon/scheduler/__init__.py", line 6, in
from .fifo import *
File "package/autogluon/scheduler/fifo.py", line 17, in
from ..searcher import BaseSearcher
File "package/autogluon/searcher/__init__.py", line 2, in
from .skopt_searcher import *
File "package/autogluon/searcher/skopt_searcher.py", line 6, in
from skopt import Optimizer
File "package/skopt/__init__.py", line 55, in
from .searchcv import BayesSearchCV
File "package/skopt/searchcv.py", line 16, in
from sklearn.utils.fixes import MaskedArray
ImportError: cannot import name 'MaskedArray'
2020-05-11 01:47:03,374 sagemaker-containers ERROR ExecuteUserScriptError:
Command "/usr/local/bin/python3.6 train.py --label y"
Traceback (most recent call last):
File "/usr/local/bin/dockerd-entrypoint.py", line 20, in
subprocess.check_call(shlex.split(' '.join(sys.argv[1:])))
File "/usr/local/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['train']' returned non-zero exit status 1.
```
**To reproduce**
1. Create the SageMaker notebook instance
- Create new role which can access any S3 bucket
- Set Git repository option to clone the amazon-sagemaker example repository to the notebook instance only.
2. Run [the notebook](https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/autogluon-tabular/AutoGluon_Tabular_SageMaker.ipynb)
Contributor guide
Research direction
Open advanced_functionality/autogluon-tabular/AutoGluon_Tabular_SageMaker.ipynb and start with the training cell that creates the Estimator and calls estimator.fit(train_s3_path). Reproduce the import failure and inspect the logged sklearn.utils.fixes MaskedArray error. Done means the notebook's training step runs without this import error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100