CosmoStat / CosmoStat/shapepipe
make_cat: objects ngmix never fit get NGMIX sentinels but MCAL_FLAGS = 0
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 18
- Forks
- 14
- Avg merge
- 8h 40m
- Merged PRs (30d)
- 10
Description
_save_ngmix_data pre-fills every NGMIX_* column with sentinels and overwrites them only for objects found in the ngmix output catalogue. An object ngmix never fit is absent from that catalogue (it raised "0 epoch to process", ngmix.py:2034, or any other exception caught at ngmix.py:1058-1063, and was skipped with continue), so it keeps every default:
NGMIX_G1/G2 = -10 make_cat.py:494
NGMIX_T, NGMIX_FLUX = 0 make_cat.py:474
NGMIX_MCAL_FLAGS = 0 make_cat.py:501
NGMIX_N_EPOCH = 0 make_cat.py:453
MCAL_FLAGS, the column meaning "the metacal fit failed", therefore reads clean for objects that were never fit at all. In final_cat_smk-g7.hdf5 (the 64-tile acceptance run of #879/#886/#887/#880) this is 18,983 of 1,851,100 objects (1.03%): cutting on MCAL_FLAGS == 0 alone gives mean e1 = −0.107 (std 1.03), against −0.004 (std 0.22) once NGMIX_N_EPOCH > 0 is also required.
Minimal fix: initialise NGMIX_MCAL_FLAGS at make_cat.py:501 to a nonzero "not fitted" value instead of 0, so the flag column is honest by default and only a real ngmix row can clear it. A dedicated bit would be preferable to a bare nonzero if the column gains a documented bit table. sp_validation's classification_galaxy_ngmix happens to reject these rows today through its NGMIX_G1_PSF_ORIG_NOSHEAR != -10 guard — an exact float comparison against a sentinel — and CosmoStat/sp_validation#343 now carries an explicit NGMIX_N_EPOCH > 0 guard instead.
— Claude (Fable) on behalf of Cail
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in make_cat.py around lines 474, 494, and 501, then trace how _save_ngmix_data fills rows missing from the ngmix output catalogue. Verify the not-fitted default for NGMIX_MCAL_FLAGS is nonzero while real ngmix rows can clear it. Check the resulting flag behavior alongside NGMIX_N_EPOCH > 0 using the final_cat_smk-g7.hdf5 acceptance output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100