Tests failing with `ValueError: cannot broadcast shape (nan,) to shape (nan,)`
- Dominant language
- Python
- Stars
- 951
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the issue**:
When updating `dask` to 2025.2.0, the following `dask-ml` tests start failing:
```
=========================== short test summary info ============================
FAILED tests/ensemble/test_blockwise.py::TestBlockwiseVotingRegressor::test_fit_array - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/ensemble/test_blockwise.py::TestBlockwiseVotingRegressor::test_fit_frame - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/linear_model/test_glm.py::test_lr_score - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/metrics/test_regression.py::test_ok[r2_score-True] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/metrics/test_regression.py::test_ok[r2_score-False] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/preprocessing/test_data.py::TestQuantileTransformer::test_fit_transform_frame - IndexError: boolean index shape (nan,) must match indexed array's (nan,).
FAILED tests/test_incremental.py::test_scoring_string[threads-r2] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/test_incremental.py::test_scoring_string[distributed-r2] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/test_parallel_post_fit.py::test_laziness - ValueError: cannot broadcast shape (nan,) to shape (nan,)
= 9 failed, 585 passed, 116 skipped, 1 deselected, 8 xfailed, 6 xpassed, 312 warnings in 204.79s (0:03:24) =
```
**Minimal Complete Verifiable Example**:
```console
$ pytest
```
**Anything else we need to know?**:
Context: bumping dask in nixpkgs: https://github.com/NixOS/nixpkgs/pull/381896
**Environment**:
- Dask version: 2025.2.0
- Python version: 3.12.8
- Operating System: NixOS
- Install method (conda, pip, source): nix
Contributor guide
Research direction
Start by running pytest with Dask 2025.2.0 and inspect the failures in tests/ensemble/test_blockwise.py, tests/linear_model/test_glm.py, tests/metrics/test_regression.py, tests/preprocessing/test_data.py, tests/test_incremental.py, and tests/test_parallel_post_fit.py. Trace the broadcast and boolean-index shape errors to determine the compatibility change, then verify that the full suite passes without regressing the reported tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100