EpistasisLab / EpistasisLab/tpot

Which pipeline should I use?

Open
#931 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I used an early stopping ("early_stop=30") in my last run, and TPOT stopped with the message:

```
The optimized pipeline was not improved after evaluating 30 more generations. Will end the optimization process.

TPOT closed prematurely. Will use the current best pipeline.

Best pipeline:

ExtraTreesClassifier(CombineDFs(FastICA(SelectPercentile(RFE(Nystroem(RFE(input_matrix,
criterion=entropy,
max_features=0.1,
n_estimators=100,
step=0.9500000000000001),
gamma=0.9,
kernel=poly,
n_components=10),
criterion=entropy,
max_features=0.55,
n_estimators=100,
step=0.35000000000000003),
percentile=76),
tol=0.35000000000000003),
input_matrix),
bootstrap=False,
criterion=gini,
max_features=0.4,
min_samples_leaf=1,
min_samples_split=3,
n_estimators=100)
```

However, the exported pipeline file contained:
```

import numpy as np
import pandas as pd
from sklearn.decomposition import FastICA
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import RFE, SelectPercentile, f_classif
from sklearn.kernel_approximation import Nystroem
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline, make_union
from tpot.builtins import StackingEstimator
from sklearn.preprocessing import FunctionTransformer
from copy import copy

# NOTE: Make sure that the class is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE', sep='COLUMN_SEPARATOR', dtype=np.float64)
features = tpot_data.drop('target', axis=1).values
training_features, testing_features, training_target, testing_target = \
train_test_split(features, tpot_data['target'].values, random_state=7)

# Average CV score on the training set was:0.7898691204650181
exported_pipeline = make_pipeline(
make_union(
make_pipeline(
RFE(estimator=ExtraTreesClassifier(criterion="entropy",
max_features=0.1,
n_estimators=100),
step=0.9500000000000001),
Nystroem(gamma=0.9,
kernel="poly",
n_components=10),
RFE(estimator=ExtraTreesClassifier(criterion="entropy",
max_features=0.55,
n_estimators=100),
step=0.35000000000000003),
SelectPercentile(score_func=f_classif,
percentile=76),
FastICA(tol=0.35000000000000003)
),
FunctionTransformer(copy)
),
ExtraTreesClassifier(bootstrap=False,
criterion="gini",
max_features=0.4,
min_samples_leaf=1,
min_samples_split=3,
n_estimators=100)
)

exported_pipeline.fit(training_features, training_target)
results = exported_pipeline.predict(testing_features)

```

I have formatted each file for the sake of readability.

My questions are:
1. Why is the exported file different?
2. Which model should I use?

Charles

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the pipeline printed in the early-stopping message with the generated export shown in the issue. Trace how TPOT represents and exports the best pipeline, then determine why the two representations differ and document which pipeline should be used.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.