astropy / astropy/ccdproc

Add sparse test backend as a densification detector (and eventual supported library)

Open
#938 0 comments 0 reactions 0 assignees View on GitHub
enhancement tests
Dominant language
Python
Stars
93
Forks
92
Avg merge
14h 44m
Merged PRs (30d)
30

Description

### Proposal

Add `sparse` (pydata/sparse) as a test backend — initially as a *diagnostic* for densification escapes, potentially graduating to a supported library (already listed as a welcome contribution in `docs/array_api.rst`).

Why: `sparse` raises `RuntimeError` on implicit densification by default (unless `SPARSE_AUTO_DENSIFY=1`), so it catches `np.asarray`-style escapes on the CPU — the same class of bug CuPy reports as `Implicit conversion to a NumPy array is not allowed`, without needing a GPU.

### Caveats

- sparse's array-API coverage is narrower than CuPy's (no general in-place mutation, gaps in reductions/sorting), so some failures will be false positives for the CuPy question. Treat a sparse *pass* as strong evidence and a sparse *failure* as needing triage — a diagnostic filter, not a CI gate, at first.
- Keep the test images dense-valued so only protocol behavior differs, not numerics.

### Work items

- [ ] Wire `sparse` into the `CCDPROC_ARRAY_LIBRARY` mechanism.
- [ ] Triage failures against the array-api-strict backend (#937): failing under both ≈ real CuPy bug.
- [ ] If coverage turns out good, add CI job + docs and promote to a supported library.

---
Follow-up to #909. Found during a review of the array API implementation from #885.

Contributor guide

Open the contributing guide

Research direction

Start by locating the CCDPROC_ARRAY_LIBRARY mechanism and the existing array-api-strict backend referenced by #937. Wire sparse in as a diagnostic backend using dense-valued test images, then triage failures against array-api-strict; done initially means the backend runs without becoming a CI gate, with CI and docs considered only if coverage is good.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.