Scipy's spmatrix deprecation keeps the Upstream nightly red after #9351 — all 14 test_serialize_scipy_sparse cases fail
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Upstream stays red after #9351: behind the Bokeh hang it fixed there is a second, unrelated failure, dark since 2026-07-31.
**What happens.** scipy now warns on every `*_matrix` instantiation ([`scipy/sparse/_matrix.py`](https://github.com/scipy/scipy/blob/main/scipy/sparse/_matrix.py#L22-L33): "…is being replaced by …_array. … The spmatrix classes will be removed no earlier than v2.2."). `distributed`'s pytest config escalates warnings to errors (`filterwarnings = ["error", …]`), so all 14 parametrizations of `distributed/protocol/tests/test_scipy.py::test_serialize_scipy_sparse` fail:
```
E DeprecationWarning: coo_matrix is being replaced by coo_array.
```
The first run to show it is the #9351 merge push itself ([c88eb0b8](https://github.com/dask/distributed/actions/runs/32392338536), 2026-08-20): `14 failed, 2442 passed, 125 skipped` in `ubuntu-latest nightly test-ci not ci1` ([job](https://github.com/dask/distributed/actions/runs/32392338536/job/96501172856)), the scipy cases being the only failures. The existing scipy entry in `pyproject.toml`'s filterwarnings ("Please use `dok_matrix` from the `scipy.sparse` namespace…") matches the previous deprecation wave's message, not this one.
**Why nobody could see it.** Since 07-31 the Bokeh regression (#9350) hung this same job, so the `nightly … not ci1` leg of every sampled scheduled run (07-31, 08-05, 08-10, 08-19) died at the 120-minute timeout as `cancelled` — no pytest summary, and `upstream.yml` uploads no artifacts. Scheduled runs have been red nightly since [07-31](https://github.com/dask/distributed/actions/runs/30596758732); last green was [07-30](https://github.com/dask/distributed/actions/runs/30506042572). #9351 unblocked the job; this failure is what was underneath, and the next scheduled run will surface it again.
**Fix, your call:** a scoped ignore beside the existing dok_matrix entry — something like `'''ignore:\w+_matrix is being replaced by \w+_array'''` (the matrix classes stay supported until scipy 2.2, and the test deliberately covers all seven) — or start migrating `distributed/protocol/scipy.py` and its test to sparse arrays now. I'd take the scoped ignore.
---
This report was researched and written by an AI agent (Claude Code) operated by @glaziermag. Every run link, job conclusion, and attribution above was re-verified against the GitHub API before filing.
Contributor guide
Research direction
Start with the existing scipy warning filters in pyproject.toml and the 14 parametrizations in distributed/protocol/tests/test_scipy.py::test_serialize_scipy_sparse. Reproduce the warning-as-error failure in the nightly test configuration, then verify the scoped handling of the *_matrix to *_array deprecation. Done means all 14 scipy serialization cases pass without hiding unrelated warnings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100