aws / aws/amazon-sagemaker-examples

Error when training and deploying xgboost in script mode

Open
#1,705 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Hi,

For a project, I want to use sagemaker xgboost to predict movie box office.
I want to use my own script for training the model and predicting.
The training data is stored in s3 as csv file.

I tried to implement my own train function by applying the example code:
https://github.com/makexu93/box-office-predictor/blob/main/train_xgboost.py

The error is:
_unexpectedstatusexception: error for training job sagemaker-xgboost-2020-11-06-19-06-38-225: failed. reason: algorithmerror: executeuserscripterror: command "/miniconda3/bin/python -m train_xgboost --eta 0.2 --gamma 4 --max_depth 5 --min_child_weight 6 --num_round 50 --objective reg:linear --silent 0 --subsample 0.7"_

Also, the predict function wont work:
https://github.com/makexu93/box-office-predictor/blob/main/predict.py

The error is:
_unexpectedstatusexception: error hosting endpoint sagemaker-xgboost-2020-11-06-10-03-24-410: failed. reason: the primary container for production variant alltraffic did not pass the ping health check. please check cloudwatch logs for this endpoint.._

Thanks for your help!

Marco

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked train_xgboost.py and predict.py files, then inspect the reported training-job and endpoint health-check errors in the SageMaker logs. Determine why script-mode training fails and why deployment does not pass its ping check; done means both training and prediction work successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.