EpistasisLab / EpistasisLab/tpot

Is creating an ensemble out of the TPOT population useful?

Open
#105 17 comments 1 reaction 0 assignees View on GitHub
enhancement question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

One of the common arguments against population-based optimization methods is that they are significantly slower than methods that work with one (or a few) solutions at a time. I think one smart way to turn that argument on its head would be to see if creating an ensemble out of the TPOT population would be useful.

An initial exploration could be to run TPOT as normal, and collect additional statistics about the performance of the population as an ensemble. This could be done with a very "hacky" version of TPOT; no need to engineer it before we prove this idea's efficacy.

Basically, for every generation:

1) Store the classifications of every individual

2) Use various ensemble methods to combine their classifications into a single classification (min, max, threshold, majority, weighted based on performance on training set)

3) Plot the effectiveness of all of these population ensemble methods over time

What to look for:
- Does the population ensemble perform better than the absolute best individual (early on, later on, always)?
- Does the population ensemble perform better as more generations pass?
- What ensemble method(s) perform best?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.