dask / dask/dask-ml

regression metrics raise exception for dask.dataframe.core.Series

Open
#756 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
951
Forks
262
PR merge metrics
No merged PRs in 30d

Description

**What happened**:

I tried to pass columns from a Dask DataFrame into regression metrics like `mean_squared_error()`, and this raised errors like

> AttributeError: 'Scalar' object has no attribute 'mean'

**What you expected to happen**:

I expected that I'd be able to pass a column from a Dask DataFrame (which has type `dask.dataframe.core.Series`) into any of the metrics functions.

**Minimal Complete Verifiable Example**:

```python
import dask
import dask.dataframe as dd
from dask.distributed import Client, LocalCluster

cluster = LocalCluster()
client = Client(cluster)
cluster

ddf = dask.datasets.timeseries()

from dask_ml.metrics import mean_squared_error

mean_squared_error(
y_true=ddf["y"],
y_pred=ddf["y"]
)
```

**Anything else we need to know?**:

I looked around and couldn't find documentation that would lead me to think this wouldn't work, or other issues that seemed related.

**Environment**:

- Dask version (output of `pip freeze | grep -E "dask|distributed"`)
- ```shell
dask==2.30.0
dask-cloudprovider==0.4.1
dask-glm==0.2.0
dask-ml==1.7.0
distributed==2.30.1
```
- Python version: `3.8.3.final.0`
- Operating System: macOS 10.14.6
- Install method (conda, pip, source): `pip`

Thanks for your time and consideration

Contributor guide

Open the contributing guide

Research direction

Start by running the minimal Dask DataFrame example with dask_ml.metrics.mean_squared_error and inspect the regression metrics entry points. Trace how dask.dataframe.core.Series inputs are handled, then verify that the metrics no longer raise the reported Scalar exception and add or update regression coverage for this case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.