EpistasisLab / EpistasisLab/tpot
Is it possible to deal with imbalanced data?
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
I have a dataset which normally has a binary class in two cases:
case 1:
label = 1 which is the minority data and important
label = -1 which is the majority data and not important
case 2:
label = 1 which is the majority data and not important
label = -1 which is the minority data and important
(the data ratio of them are normally 1:2 in case 1 and 2:1 in case 2.)
I divided the dataset into three parts, train/test for training and validation for unseen data, whatever I used the sampling to balanced to train/test data or not, it seems that the model produced by TPOT cannot produce a good result of my validation data.
- For unbalanced data, I separately used 'f1' and 'roc_auc' scoring metric, all the produced pipelines would all predict to the majority label.
- For balanced data, I separately used 'f1', 'roc_auc' and 'accuracy' scoring metric, the produced pipeline will give obviously different and inconsistent validation result among each fitting with the same pipeline.
I also applied StratifiedShuffleSplit and set the test data as 0.33 to cv parameter. And the ratio of two labels in validation data and imbalanced train/test data is the same.
Are there any scoring metrics that I can apply with to improve the problem?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.