Is an arbitrary pipeline structure useful?

One of the ideas behind TPOT is that it can create an arbitrary pipeline structure: A TPOT pipeline can have as many operators as it needs, and even perform separate analyses on copies of the data set thanks to the "Combine DFs" operator.

However, one big question remains: Is having an arbitrarily-large pipeline structure useful? Or is all we need a data preprocessor, then a feature preprocessor, then a modeling step?

We should explore this question more by taking the current version of TPOT and comparing it to a version of TPOT that fixes the pipeline structure to three steps: data preprocessing (variance threshold, standard scaler, robust scaler), feature preprocessing (polynomial features, PCA, all feature selection methods), then a modeling step (all of the models).

Perhaps we can also compare it to a four-step pipeline structure: data preprocessing (variance threshold, standard scaler, robust scaler), feature preprocessing (polynomial features, PCA), feature selection (all feature selection methods), then a modeling step (all of the models). Perhaps having feature selection as a separate step just prior to the modeling step could be useful.

In either of the "fixed pipeline structure TPOT" cases, mutations would be restricted to replacing and tuning the appropriate operators in each step. Crossover would have to be prevented from creating invalid pipelines as well. This would likely entail rolling custom mutation and crossover operators.