EpistasisLab / EpistasisLab/tpot
[Discussion] How to integrate Layered TPOT
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
Hi all,
I am working on integrating [Layered TPOT](http://ceur-ws.org/Vol-1998/paper_06.pdf) with the updated development branch, so hopefully it can be part of TPOT :) my work so far can be seen [here](https://github.com/PG-TUe/tpot/tree/LTPOT/tpot).
I had a few questions about implementation details.
### Progress bar
The progress bar shows how many individuals are evaluated out of the total amount. In layered TPOT, the time it takes to evaluate an individual is highly dependent on the layer it is evaluated (since that determines how much of data is used). That means that not every individual has the same contribution to overall progress.
We could simply calculate the total amount of individuals evaluated in the entire process, and otherwise keep the progress bar the same (update it regardless of which layer individuals are evaluated on). I don't really see this as problematic, because as is the time per individual already differs because of different components needing different times to be computed. I am not sure how the projected duration is calculated (haven't looked into that yet), but as long as it's not a moving average (or it has a sufficiently large window), the projected endtime should still be relatively accurate.
Finally, depending on verbosity additionally the best pipeline result and/or the entire pareto front is also put out during the optimization process. I would simply limit this to results obtained in the highest layer.
### Statistics of individuals
In the current version of TPOT, statistics are being tracked for each individual. Stats tracked are:
* The generation it was made in.
* The number of mutations that led to this individual.
* The number of crossovers that led to this individual.
* The parent(s) of the individual, if any.
These statistics can be left alone unmodified, though it might be interesting to have an idea in which layers the modifications happened.
I think we can compile the stats per-layer-per-generation for some of that information, and otherwise just revisit this later?
### Additional hyperparameters for TPOT
In the paper, several hyperparameters to tune Layered TPOT with are proposed:
* Number of layers
* Number of generations between transfer
* Number of individuals to transfer at once
* Whether or not to skip some generations in higher layers.
Currently, I added only one hyperparameter: the number of layers. By default, this is 1, meaning it effectively is TPOT - change it to 2 or more and you will be using Layered TPOT.
The number of generations between transfer now defaults to 2 and the number of individuals to transfer is set to be half of the population. I currently have yet to implement the skipping of generations (I decided to re-implement rather than integrate and merge - it gave me some extra clarity and I could refactor more easily).
Do we just make these hyperparameters available through the constructor?
-----------------
Looking forward to your toughts :)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.