EpistasisLab / EpistasisLab/tpot

Is it possible to deal with imbalanced data?

Open
#825 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I have a dataset which normally has a binary class in two cases:
case 1:
label = 1 which is the minority data and important
label = -1 which is the majority data and not important

case 2:
label = 1 which is the majority data and not important
label = -1 which is the minority data and important
(the data ratio of them are normally 1:2 in case 1 and 2:1 in case 2.)

I divided the dataset into three parts, train/test for training and validation for unseen data, whatever I used the sampling to balanced to train/test data or not, it seems that the model produced by TPOT cannot produce a good result of my validation data.

- For unbalanced data, I separately used 'f1' and 'roc_auc' scoring metric, all the produced pipelines would all predict to the majority label.
- For balanced data, I separately used 'f1', 'roc_auc' and 'accuracy' scoring metric, the produced pipeline will give obviously different and inconsistent validation result among each fitting with the same pipeline.

I also applied StratifiedShuffleSplit and set the test data as 0.33 to cv parameter. And the ratio of two labels in validation data and imbalanced train/test data is the same.

Are there any scoring metrics that I can apply with to improve the problem?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.