dask / dask/dask-ml

Should compute explicitly before passing X, y to sklearn scorers

Open
#248 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

I'm running through @stsievert 's hyperband example. At one point we pass dask arrays to the score method:

```python
alg = HyperBandCV(...)
...
alg.score(X_test, y_test)
```

This ends up calling an sklearn scorer here:

https://github.com/dask/dask-ml/blob/aa8467964b11f5ee5a89b5dce4dd97c75f541eba/dask_ml/model_selection/_search.py#L806

Scikit-Learn then calls np.asarray separately on X and y, leading to two independent and redundant compute calls. It would be more efficient to call

X, y = dask.compute(X, y)

rather than the implicit

X = X.compute()
y = y.compute()

especially when X and y have shared structure.

So we probably need to throw a `dask.compute` call in `DaskBaseSearchCV.score` *if* we suspect that the scorer doesn't support Dask arrays natively.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.