CosmoStat / CosmoStat/shapepipe

make_cat: objects ngmix never fit get NGMIX sentinels but MCAL_FLAGS = 0

Open Beginner friendly
#889 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
18
Forks
14
Avg merge
8h 40m
Merged PRs (30d)
10

Description

_save_ngmix_data pre-fills every NGMIX_* column with sentinels and overwrites them only for objects found in the ngmix output catalogue. An object ngmix never fit is absent from that catalogue (it raised "0 epoch to process", ngmix.py:2034, or any other exception caught at ngmix.py:1058-1063, and was skipped with continue), so it keeps every default:

NGMIX_G1/G2 = -10          make_cat.py:494
NGMIX_T, NGMIX_FLUX = 0    make_cat.py:474
NGMIX_MCAL_FLAGS = 0       make_cat.py:501
NGMIX_N_EPOCH = 0          make_cat.py:453

MCAL_FLAGS, the column meaning "the metacal fit failed", therefore reads clean for objects that were never fit at all. In final_cat_smk-g7.hdf5 (the 64-tile acceptance run of #879/#886/#887/#880) this is 18,983 of 1,851,100 objects (1.03%): cutting on MCAL_FLAGS == 0 alone gives mean e1 = −0.107 (std 1.03), against −0.004 (std 0.22) once NGMIX_N_EPOCH > 0 is also required.

Minimal fix: initialise NGMIX_MCAL_FLAGS at make_cat.py:501 to a nonzero "not fitted" value instead of 0, so the flag column is honest by default and only a real ngmix row can clear it. A dedicated bit would be preferable to a bare nonzero if the column gains a documented bit table. sp_validation's classification_galaxy_ngmix happens to reject these rows today through its NGMIX_G1_PSF_ORIG_NOSHEAR != -10 guard — an exact float comparison against a sentinel — and CosmoStat/sp_validation#343 now carries an explicit NGMIX_N_EPOCH > 0 guard instead.

— Claude (Fable) on behalf of Cail

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in make_cat.py around lines 474, 494, and 501, then trace how _save_ngmix_data fills rows missing from the ngmix output catalogue. Verify the not-fitted default for NGMIX_MCAL_FLAGS is nonzero while real ngmix rows can clear it. Check the resulting flag behavior alongside NGMIX_N_EPOCH > 0 using the final_cat_smk-g7.hdf5 acceptance output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.