aws / aws/amazon-sagemaker-examples
Issues in Training Module
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
I am facing the following error in the training part. Can someone help me with this?
UnexpectedStatusException: Error for Training job demo-xgboost-(name)-2024-02-05-11-57-23-767: Failed. Reason: AlgorithmError: framework error:
Traceback (most recent call last):
File "/miniconda3/lib/python3.6/site-packages/sagemaker_containers/_trainer.py", line 84, in train
entrypoint()
File "/miniconda3/lib/python3.6/site-packages/sagemaker_xgboost_container/training.py", line 94, in main
train(framework.training_env())
File "/miniconda3/lib/python3.6/site-packages/sagemaker_xgboost_container/training.py", line 90, in train
run_algorithm_mode()
File "/miniconda3/lib/python3.6/site-packages/sagemaker_xgboost_container/training.py", line 68, in run_algorithm_mode
checkpoint_config=checkpoint_config
File "/miniconda3/lib/python3.6/site-packages/sagemaker_xgboost_container/algorithm_mode/train.py", line 105, in sagemaker_train
train_dmatrix, val_dmatrix = get_validated_dmatrices(train_path, val_path, file_type, csv_weights, is_pipe)
File "/miniconda3/lib/python3.6/site-packages/sagemaker_xgboost_container/algorithm_mode/train.py", line 61, in get_validated_dmatrices
if train_files_
Contributor guide
Research direction
Start by reproducing the reported training job and inspect the traceback through sagemaker_xgboost_container/training.py and algorithm_mode/train.py, especially get_validated_dmatrices. The issue body is truncated before the failure details, so first identify the complete error and the training notebook or entry point involved; done means the training module completes successfully or the failure is documented with a clear fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100