EpistasisLab / EpistasisLab/tpot
Problem with warm start
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to plot the score of the best pipeline after every generation. I tried to do that with warm start. The problem is that, if I understood everything correctly, the following two examples should be identical.
I know that the number of generations and population are too low. I'm using these just for testing.
Thanks for your work and help!
Setup:
```
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data.astype(np.float64),
iris.target.astype(np.float64), train_size=0.75, test_size=0.25, random_state=42)
```
First version:
```
tpot = TPOTClassifier(generations=5, population_size=2, verbosity=2, random_state=42, warm_start=True)
tpot.fit(X_train, y_train)
```
Second version:
```
tpot2 = TPOTClassifier(generations=1, population_size=2, verbosity=2, random_state=42, warm_start=True)
for _ in range(0, 5):
tpot2.fit(X_train, y_train)
```
## Current result
The first version outputs these CV scores:
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 2 - Current best internal CV score: 0.9462450592885375
Generation 3 - Current best internal CV score: 0.9462450592885375
Generation 4 - Current best internal CV score: 0.9462450592885375
Generation 5 - Current best internal CV score: 0.9545454545454547 <--
The second example these CV scores:
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375
Generation 1 - Current best internal CV score: 0.9462450592885375 <--
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.