CosmoStat / CosmoStat/shapepipe

A/B validation of off-by-default shape-measurement options

Open
#813 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
18
Forks
14
Avg merge
8h 40m
Merged PRs (30d)
10

Description

Several ngmix-module options ship off by default, pending real-data evidence — the design contract is that production defaults flip here, on data, never in the implementing PR. Each sub-issue is one A/B question with a named metric and a named decider.

Current options awaiting a verdict:

  • BLEND_HANDLING = uberseg (#770 / #776) — UberSeg neighbour masking vs the historical noise-fill.
  • Weight symmetrisation (4-fold) — follow-on PR to #770, per @aguinot's recommendation on #776.
  • Metacal reconvolution kernel azgauss vs fiducial fitgauss (#830) — noise-robust variant of gauss, selectable via the METACAL_PSF knob added in #829 (mechanism tracked separately as #819).

The decisive statistics (B-modes, leakage) need real area, which is why this lives in the First run on data (Nibi) milestone: the A/B runs are consumers of the P3 footprint (#801, #808) — pipeline run twice (or more) with config variants only, matched catalogues compared through the sp_validation machinery. Complementary evidence from image simulations (calibrated m/c truth) rides UNIONS-WL/MultiBand_ImSim#1.

Methodology (area needed, patch choice, matched-object vs matched-area comparison) needs group design — to discuss on a tomography call before the runs are scheduled.

— Claude, on behalf of Cail

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the sp_validation machinery and the configuration variants for the listed options, then read the P3 footprint references (#801 and #808). Discuss area, patch choice, and matched-object versus matched-area comparison on a tomography call before scheduling runs. Done means real-data and simulation comparisons produce the named statistics and a group decision on each option.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.