aws / aws/amazon-sagemaker-examples
Notebook xgboost_customer_churn fails with XGBoost 1.5-1 and 1.3-1 Images
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
[xgboost_customer_churn](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_applying_machine_learning/xgboost_customer_churn/xgboost_customer_churn.ipynb)
**Describe the bug**
Notebook [xgboost_customer_churn](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_applying_machine_learning/xgboost_customer_churn/xgboost_customer_churn.ipynb) fails during training, specifically in cell starting with line of code `sess = sagemaker.Session()`. Looks like coding incompatibilities with the latest version(s).
Fails with v1.5-1 and v1.3-1 - same error (see below). Running with the older v1.2-1 gets through this issue, but may result in other issues.
**To reproduce**
Run the notebook to the training cell which fails, starting with `sess = sagemaker.Session()`.
**Logs**
If applicable, add logs to help explain your problem.
```text
[2022-08-2S 15:23:59.038 ip-10-0-198-79.ec2.internal:1 INFO hook.py:200] tensorboard_dir has not been set for the hook. SNOeb ug will not be exporting tensorboard summaries.
[2022-08-25 15:23:59.031 ip 10 8 198 79.ec2.internal:1 INFO profiler_config_parser.py:102] User has disabled profiler.
[2022-08-25 15:23:59.032 ip-10-0-198-79.ec2.internal:1 INFO hook.py:255] Saving to /opt/ml/output/tensors
[2022-88-25 15:23:59.032 ip-10-0-198-79.ec2.internal:1 INFO state_store.py:77) The checkpoint config file /opt/ml/input/confi g/checkpointconfig.json does not exist. (2022-08-25:15:23:59:INF0) Debug hook created from config
[2022-88-25:15:23:59:ERROR] Reporting training FAILURE
[2022-08-25:15:23:59:ERROR] framework error: Traceback (most recent call last): File -/miniconda3/lib/python3.8/site-packages/sagemaker_xgboost_container/algorithm_mode/train.py", line 238, in trainjob bst xgb.train(train_cfg, train_dmatrix, num_boost_round-num_round-iteration, eyals-watchlist, File "/miniconda3/11b/python3.8/site-packages/xgboost/training.py", line 188, in train bst - _train_internal(params, dtrain, File -/miniconda3/lib/python3.8/site-packages/xgboost/training.py-, line 61, in _train_internal assert all(isinstance(c, callback.Trainingallback) AssertionError: You can't mix new and old callback styles. During handling of the above exception, another exception occurred:
```
Contributor guide
Assessment
This issue has not been assessed yet.