EpistasisLab / EpistasisLab/tpot
Treat continuous hyper-parameters as being continous
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
TPOT does not distinguish between discrete and continous hyper parameters and treats everything as discrete values.
# Motivation for the issue
Many models rely on continuous parameters that can vary largely in scale depending on the dataset (for example C in a [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn-svm-linearsvc)). However, TPOT only supports describing parameters as a list of possible values and as far as I can tell, it only uses this limited list as possible values.
As a result, if users select a parameter grid that is either too wide or too narrow for his problem, then TPOT might fail to find a good solution. Even if users just use TPOT out of the box, this problem might occur.
# Proposed solutions:
There might be better ways, but I though of two approaches:
1. **Probability distribution.** Methods such as [RandomizedSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html#sklearn-model-selection-randomizedsearchcv) or Bayesian optimization methods allow users to define a probability distribution for the values.
2. **Interpolation.** When performing mutations or recombination between parents that have the same steps, continous variables could respectively receive a random noise or be interpolated from their parents. This would allow them to discover good hyperparameters in a fashion similar to Bayesian hyperoptimization.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.