alteryx / alteryx/evalml

Design - TuningComponent

Offen
#3,361 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
new feature
Vorherrschende Sprache
Python
Sterne
850
Forks
96
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

## Reasoning

It is not clear that AutoMLSearch is tuning the pipeline threshold during search.
- It's not mentioned in our docs: https://evalml.alteryx.com/en/stable/user_guide/automl.html
- It's not mentioned in the logs when `verbose=True`
image

Since it is not clear that threshold tuning is happening, users can be confused as to why the score on the best pipeline differs from the score they get when they call `fit` and `score` separately.
- See #2844

I think if threshold tuning were a component in the pipeline, it'd be very clear to users how and when threshold tuning happens during pipeline fitting. It also means users don't have to remember to tune the threshold themselves when they try to fit a pipeline "exported" from `AutoMLSearch`.

I'm interested in hearing other approaches but I think making it a component in the pipeline, similar to what we did for under/oversampling, is the best way to make it very clear that threshold tuning is taking place.

Closing Criteria:
- [ ] Binary classification pipelines have a threshold tuning component in the pipeline
- [ ] Docs describing threshold tuning

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.