autogluon / autogluon/autogluon

[BUG] Cannot set_decision_threshold before fit for TabularPredictor

Open
#3,969 2 comments 0 reactions 1 assignee Claimed by @Innixma View on GitHub
enhancement module: tabular
Dominant language
Python
Stars
10.7k
Forks
1.2k
Avg merge
21h 29m
Merged PRs (30d)
57

Description

**Bug Report Checklist**

- [x] I provided code that demonstrates a minimal reproducible example.
- [ ] I confirmed bug exists on the latest mainline of AutoGluon via source install.
- [x] I confirmed bug exists on the latest stable version of AutoGluon.

**Describe the bug**

Cannot call `TabularPredictor.set_decision_threshold` before fit, due to the logging message informing about decision threshold calibration requires the `positive_class` to be set.

**Expected behavior**

We should be able to set a decision threshold other than 0.5 for fitting, since this might change the validation score for some metrics (e.g. balanced accuracy).

**To Reproduce**

```python
from autogluon.tabular import TabularPredictor

if __name__ == "__main__":
predictor = TabularPredictor(
label="label",
problem_type="binary",
eval_metric="balanced_accuracy",
verbosity=2
)
predictor.set_decision_threshold(0.6)
```

**Screenshots / Logs**

**Installed Versions**

```python
INSTALLED VERSIONS
date : 2024-03-05
time : 09:41:32.456414
python : 3.8.10.final.0
OS : Linux
OS-release : 5.18.0-051800-generic
Version : #202205222030 SMP PREEMPT_DYNAMIC Sun May 22 20:33:46 UTC 2022
machine : x86_64
processor : x86_64
num_cores : 20
cpu_ram_mb : 64116.89453125
cuda version : None
num_gpus : 0
gpu_ram_mb : []
avail_disk_size_mb : 68699

async-timeout : 4.0.3
autogluon : None
autogluon.common : 1.0.0
autogluon.core : 1.0.0
autogluon.features : 1.0.0
autogluon.tabular : 1.0.0
boto3 : 1.33.11
catboost : 1.2.2
fastai : 2.7.13
hyperopt : 0.2.7
imodels : 1.3.18
lightgbm : 4.1.0
matplotlib : 3.7.4
networkx : 3.1
numpy : 1.24.4
onnxruntime-gpu : 1.15.1
pandas : 2.0.3
psutil : 5.9.6
ray : 2.6.3
requests : 2.31.0
scikit-learn : 1.3.2
scikit-learn-intelex: 2024.0.1
scipy : 1.10.1
setuptools : 69.0.2
skl2onnx : 1.15.0
tabpfn : 0.1.9
torch : 2.0.1+cpu
tqdm : 4.66.1
vowpalwabbit : 9.8.0
xgboost : 2.0.2
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.