EpistasisLab / EpistasisLab/tpot

Treat continuous hyper-parameters as being continous

Open
#861 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

TPOT does not distinguish between discrete and continous hyper parameters and treats everything as discrete values.

# Motivation for the issue
Many models rely on continuous parameters that can vary largely in scale depending on the dataset (for example C in a [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn-svm-linearsvc)). However, TPOT only supports describing parameters as a list of possible values and as far as I can tell, it only uses this limited list as possible values.

As a result, if users select a parameter grid that is either too wide or too narrow for his problem, then TPOT might fail to find a good solution. Even if users just use TPOT out of the box, this problem might occur.

# Proposed solutions:
There might be better ways, but I though of two approaches:
1. **Probability distribution.** Methods such as [RandomizedSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html#sklearn-model-selection-randomizedsearchcv) or Bayesian optimization methods allow users to define a probability distribution for the values.
2. **Interpolation.** When performing mutations or recombination between parents that have the same steps, continous variables could respectively receive a random noise or be interpolated from their parents. This would allow them to discover good hyperparameters in a fashion similar to Bayesian hyperoptimization.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.