Add sparse test backend as a densification detector (and eventual supported library)
- Dominant language
- Python
- Stars
- 93
- Forks
- 92
- Avg merge
- 14h 44m
- Merged PRs (30d)
- 30
Description
### Proposal
Add `sparse` (pydata/sparse) as a test backend — initially as a *diagnostic* for densification escapes, potentially graduating to a supported library (already listed as a welcome contribution in `docs/array_api.rst`).
Why: `sparse` raises `RuntimeError` on implicit densification by default (unless `SPARSE_AUTO_DENSIFY=1`), so it catches `np.asarray`-style escapes on the CPU — the same class of bug CuPy reports as `Implicit conversion to a NumPy array is not allowed`, without needing a GPU.
### Caveats
- sparse's array-API coverage is narrower than CuPy's (no general in-place mutation, gaps in reductions/sorting), so some failures will be false positives for the CuPy question. Treat a sparse *pass* as strong evidence and a sparse *failure* as needing triage — a diagnostic filter, not a CI gate, at first.
- Keep the test images dense-valued so only protocol behavior differs, not numerics.
### Work items
- [ ] Wire `sparse` into the `CCDPROC_ARRAY_LIBRARY` mechanism.
- [ ] Triage failures against the array-api-strict backend (#937): failing under both ≈ real CuPy bug.
- [ ] If coverage turns out good, add CI job + docs and promote to a supported library.
---
Follow-up to #909. Found during a review of the array API implementation from #885.
Contributor guide
Research direction
Start by locating the CCDPROC_ARRAY_LIBRARY mechanism and the existing array-api-strict backend referenced by #937. Wire sparse in as a diagnostic backend using dense-valued test images, then triage failures against array-api-strict; done initially means the backend runs without becoming a CI gate, with CI and docs considered only if coverage is good.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100