EpistasisLab / EpistasisLab/tpot

Problem with warm start

Open
#1,026 6 comments 0 reactions 0 assignees View on GitHub
bug need contributor
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I'm trying to plot the score of the best pipeline after every generation. I tried to do that with warm start. The problem is that, if I understood everything correctly, the following two examples should be identical.

I know that the number of generations and population are too low. I'm using these just for testing.

Thanks for your work and help!

Setup:
```
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data.astype(np.float64),
iris.target.astype(np.float64), train_size=0.75, test_size=0.25, random_state=42)
```
First version:
```
tpot = TPOTClassifier(generations=5, population_size=2, verbosity=2, random_state=42, warm_start=True)
tpot.fit(X_train, y_train)
```
Second version:
```
tpot2 = TPOTClassifier(generations=1, population_size=2, verbosity=2, random_state=42, warm_start=True)
for _ in range(0, 5):
tpot2.fit(X_train, y_train)
```

## Current result
The first version outputs these CV scores:
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 2 - Current best internal CV score: 0.9462450592885375
Generation 3 - Current best internal CV score: 0.9462450592885375
Generation 4 - Current best internal CV score: 0.9462450592885375
Generation 5 - Current best internal CV score: 0.9545454545454547 <--

The second example these CV scores:
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375 <--

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.