dmlc / dmlc/xgboost

xgboost.dask.DaskXGBRFRegressor.fit not handling empty dataset

Open
#11,558 3 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

##Bug

Test correctly identifies the need for explicit empty-dataset validation, and the implementation fails to raise the expected exception.
Therefore, the unit test test_fit_with_empty_dataset is correct in revealing a missing validation in the XGBoost Dask wrapper implementation.

###Reproducing Code:
```python
import unittest
import dask.array as da
from dask.distributed import Client, LocalCluster
from xgboost.dask import DaskXGBRFRegressor
import numpy as np

class TestDaskXGBRFRegressorFit(unittest.TestCase):

@classmethod
def setUpClass(cls):
# Set up a local Dask cluster
cls.cluster = LocalCluster(n_workers=2, threads_per_worker=2)
cls.client = Client(cls.cluster)

@classmethod
def tearDownClass(cls):
# Shut down the Dask cluster
cls.client.close()
cls.cluster.close()

def test_fit_with_empty_dataset(self):
# Test fitting with an empty dataset
X = da.from_array(np.empty((0, 10)), chunks=(0, 10))
y = da.from_array(np.empty(0), chunks=0)

model = DaskXGBRFRegressor(n_estimators=10)
with self.assertRaises(ValueError):
model.fit(X, y)


if __name__ == '__main__':
unittest.main()
```

###Current output:
```
[18:12:01] Task [xgboost.dask-0]:tcp://127.0.0.1:39387 got rank 0
[18:12:01] WARNING: /workspace/src/common/error_msg.cc:52: Empty dataset at worker: 0
E
======================================================================
ERROR: test_fit_with_empty_dataset (__main__.TestDaskXGBRFRegressorFit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/sjh/projects/api_guided_testgen/jiji.py", line 28, in test_fit_with_empty_dataset
model.fit(X, y)
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/xgboost/dask/__init__.py", line 2209, in fit
super().fit(**args)
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/xgboost/core.py", line 726, in inner_f
return func(**kwargs)
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/xgboost/dask/__init__.py", line 1855, in fit
return self._client_sync(self._fit_async, **args)
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/xgboost/dask/__init__.py", line 1766, in _client_sync
return self.client.sync(func, **kwargs, asynchronous=self.client.asynchronous)
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/distributed/utils.py", line 358, in sync
return sync(
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/distributed/utils.py", line 434, in sync
raise error
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/distributed/utils.py", line 408, in f
result = yield future
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/tornado/gen.py", line 769, in run
value = future.result()
File "/home/sjh/anaconda3/envs/bug_detect/lib/python3.9/site-packages/xgboost/dask/__init__.py", line 1833, in _fit_async
self._Booster = results["booster"]
TypeError: 'NoneType' object is not subscriptable

----------------------------------------------------------------------
Ran 1 test in 4.312s

FAILED (errors=1)
```

###Expected output:
Must throw appropriate error, before going in the backend, later throwing a type error.

###Version:
xgboost==2.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.