astropy / astropy/ccdproc

Array API: Combiner.sigma_clipping and the default sigma_func densify via astropy.stats

Open
#929 0 comments 0 reactions 0 assignees View on GitHub
bug combiner Upstream fix required
Dominant language
Python
Stars
93
Forks
92
Avg merge
14h 44m
Merged PRs (30d)
30

Description

### Problem

Two closely related densification points in the combining pipeline:

1. `Combiner.sigma_clipping` delegates to `astropy.stats.sigma_clip`, which internally does `np.asanyarray` and works with `np.ma` masked arrays — both densify non-numpy input:

https://github.com/astropy/ccdproc/blob/9d25eeefda8a38fc442c1a18e79ca2fa8ca40559/ccdproc/combiner.py#L449

2. `ccdproc.sigma_func` — the **default** deviation function for `median_combine` uncertainty — is built on `astropy.stats.median_absolute_deviation`, which is likewise numpy-only:

https://github.com/astropy/ccdproc/blob/9d25eeefda8a38fc442c1a18e79ca2fa8ca40559/ccdproc/core.py#L1233

These likely account for a substantial share of the `Implicit conversion to a NumPy array is not allowed` failures in #911.

### Options

- Provide xp-native implementations in ccdproc: an iterative sigma-clip using `xp.where`, and a MAD built on the fallback median discussed in #906.
- Or wait for / contribute array-API support in `astropy.stats` (tracked in #940).

---
Found during a review of the array API implementation from #885; follow-up to #909 / #911. Related: #906.

Contributor guide

Open the contributing guide

Research direction

Start in ccdproc/combiner.py at Combiner.sigma_clipping and ccdproc/core.py at sigma_func, then review the linked astropy.stats behavior and related issues #906, #911, and #940. Determine whether both densification points should gain array-API support in ccdproc or wait for astropy; done requires an agreed implementation path that avoids the reported NumPy conversion failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.