MLBazaar / MLBazaar/BTB

Pass Errors to the tuner

Open
#95 3 comments 0 reactions 1 assignee View on GitHub

@kveerama is already working on this.

Since Feb 22, 2019.

new feature
Dominant language
Python
Stars
176
Forks
40
PR merge metrics
No merged PRs in 30d

Description

Sometimes there are some hyperparameter combinations that provoke errors, so we know that those are bad values, but we don't have any score to pass to the tuner.

Should we implement a way to pass these errors to the tuners that allows them to figure out which hyperparameter values provoke them?

This is a possible implementation:

  • Allow passing None as the score, which means that score could not be computed for those hyperparameter values.
  • Filter out all training samples with score None when fitting the current estimators.
  • Implement a new binary classifier that sits before the current estimators and only predicts whether the score will be None or not, and use the predictions from this classifier to discard suggested hyperparameters during propose.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.