alteryx / alteryx/evalml

DaskEngine: Internal vs. External Clusters

Offen
#2,727 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
refactor
Vorherrschende Sprache
Python
Sterne
850
Forks
96
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

In #2667 , we added the ability of AutoMLSearch to create its own Dask LocalCluster to perform work in parallel. An important aspect of using a Dask LocalCluster is turning it off when AutoML is done with it. For LocalClusters created by AutoMLSearch, it seems reasonable that the object will shutdown the cluster after it's done. But for externally provided clusters, that might not make sense and also might not be possible.

Successful completion of this issue involves refactoring the DaskEngine such that:

1. When AutoMLSearch receives an instance as a parameter, it does not shutdown the engine. This case should simulate a consumer that is using AutoML but has access to an external Dask cluster.
2. When AutoMLSearch creates its own Engine via the convenience string, it will shutdown the engine. This case should simulate a consumer that is using AutoML on their local machine.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.