aws / aws/amazon-sagemaker-examples
[Bug Report] For XGBoost, confusion and TreeDepth rules are not working.
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
Add the link to the notebook.
**Describe the bug**
For XGBoost, confusion and TreeDepth rules are not working. We have tried using 1.2-1 and latest image of XGB.
**To reproduce**
Use any of the following -
sagemaker.image_uris.retrieve("xgboost", region, "1.2-1") OR
sagemaker.image_uris.retrieve("xgboost", region, "latest")
and then used these as rules -
rules = [
Rule.sagemaker(
base_config=rule_configs.confusion(),
rule_parameters={
"category_no": "2",
"labels": "labels",
"predictions": "predictions",
"labels_collection": "labels",
"predictions_collection": "predictions",
"min_diag": "0.9",
"max_off_diag": "0.1"
},
collections_to_save=[
CollectionConfig(
name="labels",
parameters={
"save_interval": str(save_interval)
}
),
CollectionConfig(
name="predictions",
parameters={
"include_regex": str(save_interval)
}
)
]
),
Rule.sagemaker(
base_config=rule_configs.tree_depth(),
rule_parameters={
"depth": "4"
},
collections_to_save=[
CollectionConfig(
name="trees",
parameters={
"save_interval": str(save_interval)
}
)
]
)
]
Using an estimator object.
**Logs**
Checking rule output using,
xgboost_algorithm_mode_estimator.latest_training_job.rule_job_summary()
We got the folowing error for both confusion and tree_depth rule,
'RuleEvaluationStatus': 'Error',
'StatusDetails': 'InternalServerError: We encountered an internal error. Please try again.'
Contributor guide
Research direction
Start by reproducing the issue with the shown sagemaker.image_uris.retrieve calls, confusion and tree_depth rules, and estimator rule_job_summary(). Check whether the saved collections and rule parameters produce the reported InternalServerError for both rules. Done means identifying the cause and confirming that both rule evaluations complete successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100