desihub / desihub/desispec

automate tile grouping, coadds, and redshift fitting

Open
#889 3 comments 0 reactions 2 assignees Claimed by @akremin View on GitHub
Dominant language
Python
Stars
42
Forks
26
Avg merge
1d 21h
Merged PRs (30d)
15

Description

The standard pipeline automates grouping spectra by healpix and running redshifts, but for (mini)SV we're running `desi_group_tileframes` to group by tiles/nights, and then `desi_coadd_spectra` and running `rrdesi` on that. Automate that process.

What I ran for 20200219 by hand:
```bash
#- Group by tile/night
cd /global/cfs/cdirs/desi/spectro/redux/daily
desi_group_tileframes -i exposures/20200219 -o tiles

#- Generate coadds per-spectrograph per-tile per-night
export NIGHT=20200219
for TILEID in 63075 70003 70004 70005; do
cd /global/cfs/cdirs/desi/spectro/redux/daily/tiles/$TILEID/$NIGHT
for SPECTRO in 0 3 6 7 9; do
coadd=coadd-${SPECTRO}-${TILEID}-${NIGHT}.fits
if [ ! -f $coadd ]; then
desi_coadd_spectra --coadd-cameras --nproc 8 \
-i cframe-?${SPECTRO}-*.fits \
-o coadd-${SPECTRO}-${TILEID}-${NIGHT}.fits
else
echo $coadd exists
fi
done
done

#- On a 4 node interactive job, ~1.5 min per coadd:
salloc -N 4 -C haswell -t 04:00:00 -q interactive
export NIGHT=20200219
for TILEID in 63075 70003 70004 70005; do
cd /global/cfs/cdirs/desi/spectro/redux/daily/tiles/$TILEID/$NIGHT
for SPECTRO in 0 3 6 7 9; do
zbest=zbest-${SPECTRO}-${TILEID}-${NIGHT}.fits
if [ ! -f $zbest ]; then
echo running redrock for $zbest
time srun -N 4 -n 128 -c 2 rrdesi_mpi \
-z $zbest \
-o redrock-${SPECTRO}-${TILEID}-${NIGHT}.h5 \
coadd-${SPECTRO}-*.fits
else
echo $zbest exists
fi
done
done
```

Using a single node (instead of 4) takes ~3 min per coadd, and these could be spawned as N independent single-node jobs per night in the realtime queue. The previous steps of grouping and coadding per night are fast enough they could be done on the interactive node prior to submitting the redshift jobs.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.