desihub / desihub/desispec

DESI_SPECTRO_DARK out of date

Open
#1,920 10 comments 0 reactions 1 assignee Claimed by @Waelthus View on GitHub
Dominant language
Python
Stars
42
Forks
26
Avg merge
1d 21h
Merged PRs (30d)
15

Description

@Waelthus there have been a bunch of hardware problems recently leading to new CCD configurations, and desi_spectro_dark is out of date with the result that calibrations are falling back to old $DESI_SPECTRO_CALIB darks for b4, b8, r4, z3, and z5. Example:
```
desi_preproc -n 20221123 -e 154456 --nocosmic &> preproc.log
grep "using default from" preproc.log | sort | uniq
```

To do:
* Understand why it finds a dark that it thinks it can use in $DESI_SPECTRO_CALIB but not $DESI_SPECTRO_DARK -- I understand that for very old data, but for recent config updates I would have expected both to fail, unless we are incorrectly updating darks in $DESI_SPECTRO_CALIB instead of $DESI_SPECTRO_DARK.
* Update $DESI_SPECTRO_DARK=/global/cfs/cdirs/desi/spectro/desi_spectro_dark/v2209 with darks to match all of the recent configs
* Finish migrating old $DESI_SPECTRO_CALIB darks to $DESI_SPECTRO_DARK so that it should never need to fall back to an old dark, even for early survey validation data.
* Update the code to log.error + raise exception if it can't find a matching dark in $DESI_SPECTRO_DARK.

i.e. let's get into a situation where $DESI_SPECTRO_DARK is the single correct place for it to find dark models, and if it can't find what it needs it crashes instead of falling back to a potentially wrong file. If we want to preserve a fall-back mechanism, it should fall back to something in $DESI_SPECTRO_DARK itself, plus have daily automated checks to warn us that $DESI_SPECTRO_DARK is out of date with current data (I only accidentally noticed that it was using old darks while debugging a different problem).

@julienguy heads up.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.