Errors with sklearn RandomizedGridSearch and DaskXGBoost
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
**What happened**:
When running hyperparameter search with sklearn's RandomizedGridSearch and DaskXGBoostClassifier, get the following error:
```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/utils/validation.py in _num_samples(x)
209 try:
--> 210 return len(x)
211 except TypeError as type_error:
TypeError: 'float' object cannot be interpreted as an integer
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
in
----> 1 clf.fit(X, y)
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/utils/validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, **fit_params)
757 refit_metric = self.refit
758
--> 759 X, y, groups = indexable(X, y, groups)
760 fit_params = _check_fit_params(X, fit_params)
761
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/utils/validation.py in indexable(*iterables)
297 """
298 result = [_make_indexable(X) for X in iterables]
--> 299 check_consistent_length(*result)
300 return result
301
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
257 """
258
--> 259 lengths = [_num_samples(X) for X in arrays if X is not None]
260 uniques = np.unique(lengths)
261 if len(uniques) > 1:
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/utils/validation.py in (.0)
257 """
258
--> 259 lengths = [_num_samples(X) for X in arrays if X is not None]
260 uniques = np.unique(lengths)
261 if len(uniques) > 1:
~/anaconda3/envs/daskml3/lib/python3.7/site-packages/sklearn/utils/validation.py in _num_samples(x)
210 return len(x)
211 except TypeError as type_error:
--> 212 raise TypeError(message) from type_error
213
214
TypeError: Expected sequence or array-like, got
```
**What you expected to happen**:
Return a "best_estimator" after completing the gridsearch.
Alternative approaches would be to use dask_ml.model_selection.RandomizedGridSearch(), which results in the issue reported in #758 . It's my understanding from the documentation that the DaskXGBoost object passes the Dask Array to a DMatrix for parallel training, so that parallel search in conjunction with distributed training is a challenge to implement.
**Minimal Complete Verifiable Example**:
```python
from dask.distributed import Client
import dask.dataframe as dd
import numpy as np
import xgboost as xgb
from dask_ml.compose import ColumnTransformer
import sklearn.model_selection as ms
from scipy.stats import uniform
import dask.array as da
client = Client()
client
X, y = make_classification(chunks=100)
X_col = dd.from_array(da.from_array(np.random.randint(0,2,size=X.shape[0]))).to_frame()
X = dd.from_array(X)
X = dd.concat([X, X_col], axis=1)
X = X.to_dask_array()
param_dict = {
'estimator__max_depth' : uniform(0, 1000),
'estimator__subsample' : uniform(0,100),
'estimator__colsample_bytree' : uniform(0, 10),
'estimator__n_estimators' : uniform(10, 1000),
'estimator__reg_lambda' : uniform(0, 10000),
'estimator__ccp_alpha' : uniform(0, 10000),
'estimator__gamma' : uniform(0, 20),
'estimator__scale_pos_weight' : uniform(0, 1000)
}
clf = ms.RandomizedSearchCV(xgb.dask.DaskXGBClassifier(), param_dict)
clf.fit(X, y)
```
**Anything else we need to know?**:
**Environment**:
- Dask version: 2.12 (confirmed on 2021.01.01)
- Python version: 3.7.9
- Operating System: MacOs
- Install method (conda, pip, source): conda
- distributed==2.14.0, xgboost==1.3.0, dask-ml==1.8.0
Contributor guide
Research direction
Start by reproducing the MWE with sklearn.model_selection.RandomizedSearchCV, dask_ml.model_selection.RandomizedSearchCV, and xgb.dask.DaskXGBClassifier, then compare the failure with issue #758. Trace how the search APIs validate X and y and how DaskXGBClassifier receives them; done means establishing a supported interaction that completes and returns best_estimator_, or documenting the limitation with a focused test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100