For a single record data frame train_test_split() sometimes assigns this single record to test set.
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the issue**:
_Disclaimer: I know the bug looks silly but I still wanted to give a heads up._
For a single data frame with only 1 record `train_test_split()` sometimes returns empty train set and test set with 1 record - is that desired behavior?
**Minimal Complete Verifiable Example**:
```python
import pandas as pd
import dask.dataframe as dd
from dask_ml.model_selection import train_test_split
if __name__ == '__main__':
for _ in range(20):
df = pd.DataFrame({'x0': [0], 'x1': [1], 'y': [2]})
ddf = dd.from_pandas(df, npartitions=1)
x = ddf[['x0', 'x1']]
y = ddf['y']
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3)
if x_train.shape[0].compute() == 0:
print('x_train is empty!')
break
```
**Anything else we need to know?**:
Nope
**Environment**:
- Dask version: 2023.5.0
- Dask ML version: 2023.3.24
- Python version: 3.8.15
- Operating System: Ubuntu 22.04
- Install method (conda, pip, source): pip
Contributor guide
Research direction
Start by running the issue's minimal example with dask.dataframe and dask_ml.model_selection.train_test_split, checking how often the one-record input produces an empty training set. Determine the intended behavior for a one-record split, then add or update coverage around that case and verify the split results consistently match the decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100