astropy / astropy/specutils

Implement "read_many" or similar for SpectrumList

Open
#837 3 comments 0 reactions 0 assignees View on GitHub
io
Dominant language
Python
Stars
204
Forks
134
Avg merge
12h 55m
Merged PRs (30d)
1

Description

Right now `SpectrumList` uses the same I/O scheme as all the otherloaders, where the implicit assumption is generally a *single* file is loaded into a single object. However, this is not the case for some applications of `SpectrumList`. A specific concrete examplesi the JWST MIRI spectrograph in MRS mode, which has as pipeline outputs 12 different 1d spectra that all have different shapes but are also all extracted from the same raw data exposure (effectively for this conversation, it's 12 separate detectors that cover different wavelength ranges). So it seems logical to stick them into a single `SpectrumList`.

So my proposal is adding a method to `SpectrumList` along the lines of `read_many`, which would work pretty much the same as `read` but explicitly is expecting a list of file-like objects, a list of file names, or if it's a single string it's interpreted as either a directory name or a glob pattern. Each then get individually loaded with `Spectrum1D` (which I guess `read_many` would pass ``kwargs``into), and then stuffed into the SpectrumList.

What I do *not* know is whether that runs afoul of some of the unified I/O machinery. Someone will have to try it and see I think. It may not even be critical to use the unified I/O machinery *at all* if we are just defering all the file-specific loading to `Spectrum1D`, but it's worth at least checking.

(note: this might eventually also be desirable for `SpectrumCollection` since one can image a pile of files that all have the same shape and wanting to put them into one `SpectrumCollection`. But I think that's probably best implemented instead with some way to easily "collapse" a `SpectrumList` into a `SpectumCollection`, since there's not much of a performance advantage to loading them from `SpectrumCollection` if each file has to be parsed individually anyway. At any rate, definitely a follow-on since the `SpectumList` use case is much more concrete.)

Contributor guide

Open the contributing guide

Research direction

Start by reading the SpectrumList and Spectrum1D I/O entry points, then inspect the unified I/O machinery to determine how file-like objects, filenames, directories, and glob patterns are handled. Define tests for loading multiple inputs into one SpectrumList and forwarding kwargs to Spectrum1D; done means the supported input forms work without breaking existing read behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.