can't get probability thresholds (catboost.utils.select_threshold) for multi-class (muti-target) models
- Dominant language
- C++
- Stars
- 9.1k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Problem: select_threshold(model, validation_pool, FNR=0.01) won't work for multi-class problem
catboost version: 1.1.1
Operating System: windows server 2016
CPU: yes
GPU: no
so this video https://youtu.be/usdEWSDisS0?t=3199 London2019 notebook from examples can be found in catboost repository
My code has multiple targets (more than 1 class). It works, however it won't work with getting thresholds for better results as shown on the video above (note the timing in the url)
# I'm using threshold functions from below declarations
from catboost.utils import get_roc_curve
from catboost.utils import get_fpr_curve
from catboost.utils import get_fnr_curve
from catboost.utils import select_threshold
from catboost import CatBoostClassifier

My solution is simply to use 1 class per model (thus ending up with multiple models), but then the whole beauty of catboost is that it can do multiclassification. Also, can I assume that doing it this way I could get better results because individual models can be more complex or it's a wrong intuition? Pls advise.
PS: Btw, suppose I do multiple model files (each has different targets) and will identify needed thresholds for each target. Can I then simply "stick" those models together and use thresholds acquired before? Would the resulting model switch from classification to multi-classification automatically?
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.