[wishlist]: Read calibration data from pypeit instead of downloading it
- Dominant language
- Jupyter Notebook
- Stars
- 69
- Forks
- 43
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 3
Description
Hi,
While preparing Guix Astro Update 2025/11 and refreshing python-specreduce to 1.7.0 I've noticed
that it tried to download calibration data from GitHub for one test:
```
def load_pypeit_calibration_lines(
lamps: Sequence | None = None,
wave_air: bool = False,
cache: bool | Literal['update'] = True,
show_progress: bool = False
) -> QTable | None:
"""
Load reference calibration lines from ``pypeit`` linelists. The ``pypeit`` linelists are
well-curated and have been tested across a wide range of spectrographs. The available
linelists are defined by ``PYPEIT_CALIBRATION_LINELISTS``.
Parameters
----------
lamps : Lamp string, comma-separated list of lamps, or sequence of lamps to include in
output reference linelist. The parlance of "lamp" is retained here for consistency
with its use in ``pypeit`` and elsewhere. In several of the supported cases the
"lamp" is the sky itself (e.g. OH lines in the near-IR).
The available lamps are defined by ``PYPEIT_CALIBRATION_LINELISTS``.
wave_air : If True, convert the vacuum wavelengths used by ``pypeit`` to air wavelengths.
cache : Toggle caching of downloaded data
show_progress : Show download progress bar
Returns
-------
linelist:
Table containing the combined calibration line list. ``pypeit`` linelists have the
following columns:
* ``ion``: Ion or molecule generating the line.
* ``wavelength``: Vacuum wavelength of the line in Angstroms.
* ``NIST``: Flag denoting if NIST is the ultimate reference for the line's wavelength.
* ``Instr``: ``pypeit``-specific instrument flag.
* ``amplitude``: Amplitude of the line. Beware, not consistent between lists.
* ``Source``: Source of the line information.
"""
if lamps is None:
return None
if not isinstance(lamps, Sequence):
raise ValueError(f"Invalid calibration lamps specification: {lamps}")
if isinstance(lamps, str):
if ',' in lamps:
lamps = [lamp.strip() for lamp in lamps.split(',')]
else:
lamps = [lamps]
linelists = []
for lamp in lamps:
if lamp in PYPEIT_CALIBRATION_LINELISTS:
data_url = f"{PYPEIT_DATA_URL}/arc_lines/lists/{lamp}_lines.dat"
try:
data_path = download_file(data_url, cache=cache,
show_progress=show_progress,
pkgname='specreduce')
linelists.append(Table.read(data_path, format='ascii.fixed_width', comment='#'))
except URLError as e:
> warnings.warn(f"Downloading of {data_url} failed: {e}", AstropyUserWarning)
E astropy.utils.exceptions.AstropyUserWarning: Downloading of https://raw.githubusercontent.com/pypeit/pypeit/release/pypeit/data//arc_lines/lists/ArI_lines.dat failed:
specreduce/calibration_data.py:167: AstropyUserWarning
=========================== short test summary info ============================
FAILED specreduce/tests/test_wavecal1d.py::test_init_line_list - astropy.utils.exceptions.AstropyUserWarning: Downloading of https://raw.githubusercontent.com/pypeit/pypeit/release/pypeit/data//arc_lines/lists/ArI_lines.dat failed:
================= 1 failed, 104 passed, 24 deselected in 6.35s =================
```
However `pypeit` provides data in PyPI package, see https://github.com/pypeit/PypeIt/tree/1.17.4/pypeit/data.
May it be helpful to have some environment variable to check for calibration data from or try to read it from the `pypeit` package?
Thanks,
Oleg
Contributor guide
Research direction
Start in specreduce/calibration_data.py at load_pypeit_calibration_lines and inspect the download path around line 167, then review specreduce/tests/test_wavecal1d.py::test_init_line_list. Check how calibration files are exposed by the pypeit package and decide how local package data or the proposed environment-variable source should be selected. Done means the test can load the ArI calibration data without requiring a network download.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100