EpistasisLab / EpistasisLab/tpot

FeatureUnion + StackingEstimator causes input data to be duplicated for the rest of the model, increasing computational load and complexity.

Open
#1,242 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

TPOT uses FeatureUnion to combined the outputs of multiple operators. However, it is possible for tpot to put in two stacking estimators within a FeatureUnion block. This causes tpot to pass along two identical copies on the dataset into the next operator.

## Context of the issue

This increases computational load and complexity, especially for large datasets, with no benefit. It may also have a performance impact on certain models.

## Process to reproduce the issue

1. User creates TPOT instance
2. User calls TPOT `fit()` function with training data
3. TPOT will generate a pipeline as described.

To demonstrate the issue, below is code using a pipeline that was found by tpot.

```
from sklearn.pipeline import FeatureUnion, Pipeline
from tpot.builtins import StackingEstimator, ZeroCount
from sklearn.ensemble import RandomForestRegressor, ExtraTreesRegressor
from sklearn.linear_model import SGDRegressor
from sklearn.decomposition import PCA
import numpy as np

p = Pipeline(
[('featureunion', FeatureUnion(transformer_list=[('stackingestimator-1',
StackingEstimator(estimator=RandomForestRegressor(max_features=0.45,
min_samples_leaf=9,
min_samples_split=4))),
('stackingestimator-2',
StackingEstimator(estimator=ExtraTreesRegressor(max_features=0.7500000000000001,
min_samples_leaf=20,
min_samples_split=18)))])), ('stackingestimator-1', StackingEstimator(estimator=SGDRegressor(alpha=0.01, eta0=1.0,
fit_intercept=False, l1_ratio=0.0,
loss='epsilon_insensitive',
penalty='elasticnet', power_t=1.0))), ('pca', PCA(iterated_power=3, svd_solver='randomized')), ('stackingestimator-2', StackingEstimator(estimator=SGDRegressor(alpha=0.001, fit_intercept=False,
l1_ratio=0.0,
loss='epsilon_insensitive',
penalty='elasticnet', power_t=1.0))), ('zerocount', ZeroCount()), ('sgdregressor', SGDRegressor(alpha=0.001, fit_intercept=False, l1_ratio=0.5,
learning_rate='constant', loss='huber', penalty='elasticnet',
power_t=0.1))]
)

X = np.random.rand(5,10)
y = np.random.rand(5)

p.fit(X,y)

xx = [range(10)]
print("Input data " ,xx)
print("After featureUnion ", p.steps[0][1].transform(xx))
```

## Expected result

The data should not be copied over twice.

[Estimator 1 predictions, Estimator 2 predictions, X]

[0.44, .45, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

## Current result

[Estimator 1 predictions, X, Estimator 2 predictions, X]

[0.44, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, .45, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

## Possible fix

Here is my idea off the top of my head:
Limit featureUnion to selectors, transformers, and at most one classifier or regressor. That way only one copy of the data exists. When more than one classifier or regressor is used, replace the featureUnion with the sklearn stackingclassifier or stackingregressor. These functions similarly allow multiple models to pass along their predictions, but then only pass forward one copy of the dataset.

https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingClassifier.html
https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingRegressor.html

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.