CosmoStat / CosmoStat/shapepipe

ngmix: the reduced-χ² anomaly on stars (error model)

Open
#769 0 comments 0 reactions 2 assignees View on GitHub

@fabianhervaspeters is already working on this.

Since Jun 30, 2026.

Dominant language
Python
Stars
18
Forks
14
Avg merge
8h 40m
Merged PRs (30d)
10

Description

On stars, reduced-χ² runs too large at the bright end (climbing to ~8 for the brightest) and too small at the faint end (~0.2, against an expected ~1). χ² naturally improves for faint objects (noise dominates the denominator), but ~0.2 is well below 1 and points at a mis-estimated error model or an over-flexible fit. The faint-star ≈0.2 is the genuinely concerning piece: it implies the assumed variance is ~5× too large (σ inflated ~2.2×).

The sign flips with brightness, so a single noise-mis-scaling can't explain both tails — the answer likely names more than one effect:

  • bright-high reads as the Gaussian model genuinely failing to represent a near-point source through an imperfect PSF model (model inadequacy at the deconvolution singularity — consistent with the star-response R-climb floor);
  • faint-≈0.2 reads as the assumed errors being too large — which ties to the weight map / noise estimate fed to ngmix and to the #749 prior/weight thread.

Checkable: χ²_red vs magnitude re-derived from a controlled rerun, split by branch; the noise/weight path the fit actually consumes audited end-to-end (prepare_ngmix_weights ngmix.py:986, get_noise ngmix.py:941); and, where fixable, the assumed errors corrected so χ²_red → ~1, or the residual named as an intrinsic model-inadequacy floor.

χ² is not exported today — the fit result dict surfaces only s2n (ngmix.py:505–512); χ² lives inside the ngmix LM result. A reduced-χ² column needs wiring out per object (the prototype feat(ngmix): write reduced chi2 …, commit ba384428 on investigate/star-response, is the clean seed). That export is what unblocks the χ²-vs-magnitude panel the verification suite (#765) wants. Also fix the χ²-vs-mag plot's "Galaxy Fit" y-axis mislabel — it sits on a star fit.

Longer-term (not blocking the ngmix v2.0 merge — under #773). Coupled to the dropped PSF noise-injection (its own issue under #762), which is a leading candidate for the error-model mis-specification this anomaly measures. Related: #749, #765 (consumes the χ² export).

— Claude on behalf of Cail

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.