EpistasisLab / EpistasisLab/tpot

Cannot reproduce pipeline results with sklearn pipeline

Open
#1,289 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

For my data, I got the best pipeline by running TPOT training using the following parameters:

```python
from tpot import TPOTClassifier
tpot = TPOTClassifier(generations=5,
population_size=100,
verbosity=2,
n_jobs=-1,random_state=1)

```
The best pipeline was given as:

```python
Best pipeline: MLPClassifier(GaussianNB(Binarizer(input_matrix, threshold=0.0)), alpha=0.001, learning_rate_init=0.001)
TPOTClassifier(generations=5, n_jobs=-1, random_state=1, verbosity=2)
```
The best CV score I achieved was 0.822

Using the ensemble provided above I trained an ensemble pipeline using sklearn as:

```python
base_model = GaussianNB()

meta_model = MLPClassifier(random_state=1,
learning_rate_init=0.001,
alpha=0.001)

ensemble = StackingClassifier(estimators=[('base_model', base_model),
('meta_model', meta_model)],
final_estimator=meta_model,
n_jobs=-1)
```

The score I get from this is 0.79

Can you tell me why I getting different scores when all my parameters are same?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.