EpistasisLab / EpistasisLab/tpot
Smart seeding of TPOT populations?
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
Sorry if the text below sounds like rambling -- I was using this issue to brainstorm.
I've been thinking about possible ways to make TPOT perform better right out of the box, without having to run it for several generations to finally discover the better pipelines. One of the ideas I've had is to seed the TPOT population with a smarter group of solutions.
For example, we know that a TPOT pipeline will need at least one model, so we can seed it with each of the 6 current models over a small range of parameters:
- decision tree: all combinations of
- max_features = [0 (--> auto), 1 (--> None)]
- max_depth: [0 (--> None), 1, 5, 10, 20, 50]
- = 12 total combinations
- random forest: all combinations of
- n_estimators = [100, 500]
- max_features = [0 (--> auto), 1]
- = 4 total combinations
- logistic regression:
- C = [0.01, 0.1, 0.5, 1.0, 10.0, 50.0, 100.0]
- = 7 total combinations
- svc:
- C = [0.01, 0.1, 0.5, 1.0, 10.0, 50.0, 100.0]
- = 7 total combinations
- knnc:
- n_neighbors = [2, 5, 10, 20, 50]
- = 5 total combinations
- gradient boosting: all combinations of
- learning_rate: [0.01, 0.1, 0.5, 1.0]
- n_estimators: [100, 500]
- max_depth: [0 (--> None), 5, 10]
- = 24 total combinations
That gives us 59 "classifier-only" TPOT pipelines to start with.
We also have 4 feature selectors:
- RFE: all combinations of
- num_features = [1, 5, 10, 50]
- step = [0.1, 0.25, 0.5]
- = 12 total combinations
- select percentile:
- percentile = [1, 5, 10, 25, 50, 75]
- = 6 total combinations
- select k best:
- k = [1, 2, 5, 10, 20, 50]
- = 6 total combinations
- variance threshold:
- threshold = [0.1, 0.2, 0.3, 0.4, 0.5]
- = 5 total combinations
And 4 feature preprocessors:
- standard scaler (no parameters)
- = 1 total combinations
- robust scaler (no parameters)
- = 1 total combinations
- polynomial features (no parameters)
- = 1 total combinations
- PCA:
- n_components = [1, 2, 4, 10, 20]
- = 5 total combinations
Thus, if we wanted to provide at least one feature preprocessor _or_ selector in the pipeline before passing the data to the model, that would result in:
feature selection combinations = 12 \* 59 + 6 \* 59 + 6 \* 59 + 5 \* 59 = 1,711
feature preprocessor combinations = 5 \* 59 + 1 \* 59 + 1 \* 59 + 1 \* 59 = 472
Giving us a total = 59 + 1,711 + 472 = 2,242 pipeline combinations to start out with.
We'd evaluate all 2,242 of these pipelines then use the top 100 to seed the TPOT population. From there, the GP algorithm is allowed to tinker with the pipeline, fine-tune the parameters, and possibly discover better combinations of pipeline operators.
That's obviously a lot of pipelines to try out at the beginning -- about 23 generations worth of pipelines, which will be quite slow on any decently sized data set. It may be necessary to cut down on the parameters that we try out at the beginning.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.