EpistasisLab / EpistasisLab/tpot

cv in TPOTClassifier

Open
#1,163 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I faced an issue with tpot:
I am working with ~70000 x 36 features and predicting a multi-class target.
In particular, I am playing with Tanzania water wells [dataset](https://www.drivendata.org/competitions/7/pump-it-up-data-mining-the-water-table/)
Target is imbalanced :
functional 0.54
non functional 0.38
functional needs repair 0.07

To process such data in a reasonable time, I am using the tpot - cuml configuration.

I created TPOTClassifier object with the following parameters:

```
tpt=TPOTClassifier(generations=100, population_size=100,
offspring_size=None, mutation_rate=0.9,
crossover_rate=0.1,
scoring='accuracy', cv=3,
subsample=1.0, n_jobs=1,
max_time_mins=None, max_eval_time_mins=5,
random_state=None, config_dict='TPOT cuML',
template=None,
warm_start=False,
memory=None,
use_dask=False,
periodic_checkpoint_folder=PREFIX+'tpot_checkpoints',
early_stop=None,
verbosity=3,
disable_update_check=False,
log_file=PREFIX+'tpot_checkpoints/log_cuml.txt',

)
```

After a few hundreds of iterations, it falls with traceback:
```
/usr/local/lib/python3.6/site-packages/sklearn/model_selection/_split.py:668: UserWarning: The least populated class in y has only 2 members, which is less than n_splits=3.
% (min_groups, self.n_splits)), UserWarning)
```

I believe that this kind of traceback indicates that there is no minor class present in one of the splits.

Can you please confirm that when I am setting cv= 3,5, 10 - tpot uses stratified folds?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.