dask / dask/dask-expr

DataFrame.divisions are lost on repartition when npartions==1

Open
#975 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
89
Forks
26
PR merge metrics
No merged PRs in 30d

Description

DataFrame.divisions are lost when using `repartition` or `set_index` with `npartitions == 1`

```python
df = pd.DataFrame(np.random.randint(0, 20000, size=(10, 3)), columns=list('ABC'))
ddf = dd.from_pandas(df)
print(ddf.divisions) # (0, 9)

ddf = ddf.reset_index().set_index("A", sort=True, npartitions=2)
print(ddf.divisions) # (1483, 19649)

ddf = dd.from_pandas(df)
ddf = ddf.reset_index().set_index("A", sort=True, npartitions=1)
print(ddf.divisions) # (None, None)
```

**Environment**:

- Dask version: `2024.3.0`
- Python version: `3.12.2`
- Operating System: Osx
- Install method (conda, pip, source): pip

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the Python snippet using repartition or set_index with npartitions=1, then trace the relevant dask-expr implementation paths for those operations. Done means the resulting DataFrame.divisions are preserved rather than reported as (None, None), while the existing multi-partition behavior remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.