CosmoStat / CosmoStat/sp_validation

Retire patch logic — ShapePipe v2 has no patches

Open
#340 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2
Forks
5
Avg merge
1d 14h
Merged PRs (30d)
19

Description

ShapePipe's Snakemake workflow (CosmoStat/shapepipe#852) processes a campaign — a tile list — with no intermediate "patch" (P1…P7) unit. Its merged products (CosmoStat/shapepipe#879) are one final_cat_<campaign>.hdf5 and one full_starcat-0000000.fits per campaign. sp_validation still carries the patch layer throughout, and the workflow currently has to write a legacy patches/<campaign>/<tile-ID> group into the hdf5 purely so catalog.py's reader opens it.

Where patch assumptions live today (rough grep, jackknife npatch excluded):

  • src/sp_validation/catalog.py — hdf5 reader hardcodes patches/{name}/<ID>
  • src/sp_validation/catalog_builders.pyget_patches() (v1/v1.5P1..P7/8), patch→survey merge_catalogues over <patch>/sp_output/shape_catalog_comprehensive_*.fits
  • src/sp_validation/survey.py — patch boundary coordinates
  • src/sp_validation/rho_tau.py, cosmo_val/real_space.py — patch-keyed paths/labels
  • scripts: combine_results.py, merge_psf_cat.py, prepare_patch_for_spval.sh, plot_rho_stats_patches.py, survey_stats_all.sh, stats_tile_id_gal_counts.py, star_match_stats.py, check_tile_IDs_SP_LF.py, compute_area.py

Proposed end state: sp_validation takes a campaign's two merged files directly (hdf5 with a flat per-tile layout, or a single top-level group; star cat as-is), and any cross-campaign combination is expressed as "list of campaigns", not "list of patches". Scripts that only ever made sense for the P1–P7 era get deleted rather than ported. Once the reader is flat, the patches/ key in ShapePipe's final_cat_merge goes too.

— Claude (Fable) on behalf of Cail

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/sp_validation/catalog.py and catalog_builders.py, then trace patch-keyed paths through survey.py, rho_tau.py, cosmo_val/real_space.py, and the listed scripts. Compare the reader and merge flow with the two campaign-level merged files; done means patch-only scripts are removed or retired, campaign lists replace patch lists, and ShapePipe no longer writes the legacy patches/ key.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.