aws / aws/amazon-sagemaker-examples

[Bug Report] XGBoost problem in ml_ops/sm-mlflow_pipelines/sm-mlflow_pipelines.ipynb

Open Beginner friendly
#4,823 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Link to the notebook**
[ ml_ops/sm-mlflow_pipelines/sm-mlflow_pipelines.ipynb](https://github.com/aws/amazon-sagemaker-examples/blob/default/%20%20%20ml_ops/sm-mlflow_pipelines/sm-mlflow_pipelines.ipynb)

**Describe the bug**
Pipeline execution fails due to two different version installations of XGboost. One by Conda from the Sagemaker Distribution Image. The other from Pip in `requirements.txt`.

error log in CloudWatch:
```
* XGBoost is first installed with anaconda then upgraded with pip. To fix it please remove one of the installations.
```

**To reproduce**
Run the notebook on SageMaker Studio.

**Fix**
1. remove version number for XGBoost in `requirements.txt`
`xgboost==1.7.6` -> `xgboost`

2. move `early_stopping_rounds=5` to the instanciation of the XGBClassifier
```
xgb = XGBClassifier(n_estimators=num_round, early_stopping_rounds=5, **param)
xgb.fit(
train_df,
y_train,
eval_set=[(validation_df, y_validation)]
)
```

Contributor guide

Open the contributing guide

Research direction

Open ml_ops/sm-mlflow_pipelines/sm-mlflow_pipelines.ipynb and its referenced requirements.txt, then inspect the XGBoost installation and XGBClassifier setup described in the issue. Apply the two requested changes and rerun the notebook on SageMaker Studio; done means the pipeline executes without the duplicate-installation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.