dask / dask/dask-ml

Models without built-in or custom scoring cannot be wrapped with Incremental

Open
#925 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

Hello,
I hope you are all doing well!

Currently, models that do not have built-in scoring cannot be used with dask_ml wrappers, unless custom scoring is specified. For example, using this code, taken mostly from the sklearn example [here](https://scikit-learn.org/stable/auto_examples/cluster/plot_birch_vs_minibatchkmeans.html#sphx-glr-auto-examples-cluster-plot-birch-vs-minibatchkmeans-py)

```
import numpy as np
import dask.array as da
from sklearn.cluster import Birch
from dask_ml.wrappers import Incremental
from sklearn.datasets import make_blobs

xx = da.linspace(-22, 22, 10)
yy = da.linspace(-22, 22, 10)
xx, yy = da.meshgrid(xx, yy)
n_centers = da.hstack((da.ravel(xx)[:, np.newaxis], da.ravel(yy)[:, np.newaxis]))

X, y = make_blobs(n_samples=25000, centers=n_centers, random_state=0)
birch = Birch(threshold=1.7, n_clusters=None)
inc = Incremental(birch)
inc.fit(X)
```

This code currently, raises a TypeError because: If no scoring is specified, the estimator passed should have a 'score' method. The estimator Birch(n_clusters=None, threshold=1.7) does not.

I fixed it within Pull Request [#923](https://github.com/dask/dask-ml/pull/923#issue-1216814466) by allowing for None when calling check_scoring within the fit method. If there is a reason to not allow a loosening of this check, I am open to other solutions. Other potential solutions I saw at a glance include:

adding a parameter to fit that propagates to the allow_none value of check_scoring - defaulted to False
removing the line, as I do not see scoring being used directly in _fit_for_estimator, and other areas of the code explicitly check for/handle scoring = None.

**Environment**
OS - CentOS-7
python - v3.8.2
dask - v2022.04.1
dask-ml - v2022.01.22
sklearn - v1.0.2

Thanks!
Nick

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.