Flaky `test_collection.py::test_len`
- Dominant language
- Python
- Stars
- 13.9k
- Forks
- 2k
- PR merge metrics
- No merged PRs in 30d
Description
This test fails on both pandas 2.3 and 3.0 CI with some frequency:
https://github.com/dask/dask/actions/runs/21682277456/job/62519785526?pr=12271
```
def test_len(df, pdf):
df2 = df[["x"]] + 1
assert len(df2) == len(pdf)
assert len(df[df.x > 5]) == len(pdf[pdf.x > 5])
first = df2.partitions[0].compute()
> assert len(df2.partitions[0]) == len(first)
E AssertionError: assert 100 == 10
E + where 100 = len(Dask DataFrame Structure:\n x\nnpartitions=1 \n0 int64\n10 ...\nDask Name: partitions, 4 expressions\nExpr=Partitions(frame=df[['x']] + 1, partitions=[0]))
E + and 10 = len( x\n0 1\n1 2\n2 3\n3 4\n4 5\n5 6\n6 7\n7 8\n8 9\n9 10)
dask/dataframe/dask_expr/tests/test_collection.py:1718: AssertionError
FAILED dask/dataframe/dask_expr/tests/test_collection.py::test_len - AssertionError: assert 100 == 10
+ where 100 = len(Dask DataFrame Structure:\n x\nnpartitions=1 \n0 int64\n10 ...\nDask Name: partitions, 4 expressions\nExpr=Partitions(frame=df[['x']] + 1, partitions=[0]))
+ and 10 = len( x\n0 1\n1 2\n2 3\n3 4\n4 5\n5 6\n6 7\n7 8\n8 9\n9 10)
```
Contributor guide
Research direction
Start with dask/dataframe/dask_expr/tests/test_collection.py::test_len and reproduce the failure on the pandas 2.3 and 3.0 CI environments referenced in the issue. Compare the lazy partition length with the computed partition length around the failing assertion. Done means the test is stable and the partition length assertion consistently matches the computed result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100