Parallelization with python dask and dask-learn. Proposal.

According to Issue https://github.com/rhiever/tpot/issues/177 I would like to propose work on the distributed/cluster environment. I believe that would be a major advancement on the applicability of **tpot** to real world problems. At this moment I am working with a dataset roughly (200000 , 500) and it smashes one core while the rest sit idle.

There has been discussion on python-dask [1], which is for the unfamiliar something like TensorFlow. You symbolically build a graph of the operations and then you call the _compute_ method. This computation is automatically distributed, and intermediate results cached. I'd recommend taking a look at the official documentation (it took me 20 minutes on a train ride, so no big deal) to see some examples [2-5].

The first step I would like to propose is to model a single individual (sklearn pipeline) as a graph. Some work has been carried out in the Dask-learn project [6]. This project builds a graph for a single Pipeline.

Next step would be to model the entire population for each generation as a graph. As far as I know there is no work in this direction, but it _should_ not be particularly complex to generalize. Since many individuals share many components, Dask will automatically execute them once and reuse the results for the others, dramatically increasing performance. At the same time it distributes computations between many cores in a single or multiple mode architecture.

At this point the whole generation will be calculated at the same time.

The advantages are clear, mainly (a) distributed computation and (b) caching of intermediate results.
As disadvantages I see:
- If an individual takes x1000 times more than the rest, the system will be mostly idle waiting for it to finish. That can be partially solved by allowing an individual to use many cores, but that opens a new set of problems (like potentially having more threads running than cores).
- Intermediate states of the graph are not cached _between_ generations. The solution for this is trickier. Maybe it is possible to build a graph for **all** generations at the same time, but that would imply to reimplement DEAP as a dask graph and just the idea makes my head spin.

I'd love to hear your comments/considerations. As said above I believe this kind of unlimited scalability is worth the effort and a large part of the work has been already done by the _dask-learn_ project. In addition, development of this new feature can be orthogonal to the development of core features.

Regards,

Eduardo

[1] http://dask.pydata.org/en/latest/
[2] https://github.com/dask/dask-tutorial/blob/master/01-Array.ipynb
[3] https://github.com/dask/dask-tutorial/blob/master/02-Foundations.ipynb
[4] https://github.com/dask/dask-tutorial/blob/master/03a-DataFrame.ipynb
[5] https://github.com/dask/dask-tutorial/blob/master/04-Imperative.ipynb
[6] http://blaze.pydata.org/blog/2015/10/19/dask-learn/