Should compute explicitly before passing X, y to sklearn scorers
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
I'm running through @stsievert 's hyperband example. At one point we pass dask arrays to the score method:
```python
alg = HyperBandCV(...)
...
alg.score(X_test, y_test)
```
This ends up calling an sklearn scorer here:
https://github.com/dask/dask-ml/blob/aa8467964b11f5ee5a89b5dce4dd97c75f541eba/dask_ml/model_selection/_search.py#L806
Scikit-Learn then calls np.asarray separately on X and y, leading to two independent and redundant compute calls. It would be more efficient to call
X, y = dask.compute(X, y)
rather than the implicit
X = X.compute()
y = y.compute()
especially when X and y have shared structure.
So we probably need to throw a `dask.compute` call in `DaskBaseSearchCV.score` *if* we suspect that the scorer doesn't support Dask arrays natively.
Contributor guide
Assessment
This issue has not been assessed yet.