EpistasisLab / EpistasisLab/tpot2

Make the cv early stop functions into standalone functions? Or remove?

Open
#69 1 comment 0 reactions 0 assignees View on GitHub
API enhancement question
Dominant language
Jupyter Notebook
Stars
250
Forks
33
PR merge metrics
No merged PRs in 30d

Description

Currently, these are built into the evolver class. If we merge the revolver classes, this may be easier to use as a standalone function #65

The two strategies for pruning CV evaluation early are found in the `evaluate_population_selection_early_stop` function of the baseevolver. They are controlled by the following parameters, which may be confusing.

```
threshold_evaluation_early_stop = None,
threshold_evaluation_scaling = .5,
min_history_threshold = 20,
selection_evaluation_early_stop = None,
selection_evaluation_scaling = .5,
evaluation_early_stop_steps = None,
final_score_strategy = "mean",
```

It might be easier to pull these out and turn them into individual functions outside of the class, similar to the optuna pruning API.

But also, this feature may not be useful given that we already support successive halving. We could evaluate each independently and both together. If successive halving gives the same performance improvements, it is possible we could drop this feature to simplify the code.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.