DaskXGBClassifier and dask-ml GridSearchCV throws TypeErrors with Dask Arrays.
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
I believe this is related to #521. When using dask-ml with `xgboost.dask.DaskXGBClassifier` and `dask_ml.model_selection.GridSearchCV`, you run into TypeErrors suggesting that the estimator expects `dask.array.core.Array ` types, even while using Dask Arrays.
MCVE:
```python
import dask.array as da
import dask.dataframe as dd
import numpy as np
import pandas as pd
from dask_ml.model_selection import GridSearchCV
from xgboost.dask import DaskXGBClassifier
from dask.distributed import Client
from sklearn.datasets import load_iris
if __name__ == '__main__':
client = Client()
data = load_iris()
x = da.from_array(data.data)
y = da.from_array(data.target)
estimator = DaskXGBClassifier(objective='multi:softmax', num_class=4)
grid_search = GridSearchCV(
estimator,
param_grid={
'n_estimators': np.arange(15, 105, 15)
},
)
grid_search.fit(x, y)
results = pd.DataFrame(grid_search.cv_results_)
print(results.to_string())
```
Error:
```
distributed.worker - WARNING - Compute Failed
Function: fit_and_score
args: (DaskXGBClassifier(num_class=4, objective='multi:softmax'), , array([[5.1, 3.5, 1.4
, 0.2],
[4.9, 3. , 1.4, 0.2],
[4.7, 3.2, 1.3, 0.2],
[4.6, 3.1, 1.5, 0.2],
[5. , 3.6, 1.4, 0.2],
[5.4, 3.9, 1.7, 0.4],
[4.6, 3.4, 1.4, 0.3],
[5. , 3.4, 1.5, 0.2],
[4.4, 2.9, 1.4, 0.2],
...
kwargs: {}
Exception: TypeError("Expecting or . Got ")
```
**Environment**:
- Dask version: 2021.04.0
- Dask-ml version: 1.9.0
- Python version: 3.8.5
- Operating System: Ubuntu 20.04
- Install method (conda, pip, source): pip
Contributor guide
Research direction
Start by running the supplied MCVE with dask_ml.model_selection.GridSearchCV and xgboost.dask.DaskXGBClassifier to reproduce the TypeError. Trace the GridSearchCV fit path where the Dask Array becomes a NumPy array; done means the example completes and prints cv_results_ without the type error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100