Serialization of Spectrum1D object (How to make Spectrum1D pickle-able?)
- Dominant language
- Python
- Stars
- 204
- Forks
- 134
- Avg merge
- 12h 55m
- Merged PRs (30d)
- 1
Description
Hello everyone,
I struggle with serialization (through the `pickle` or `dill` library) of the `sp.Spectrum1D` object. In particular, the default wcs is not pickle-able. I use serialization to both save the data as pickle file and for `multiprocessing` of several functions, so this functionality would be helpful, especially since I actually don't really care about the `sp.Spectrum1D.wcs` attribute.
I know it's possible to bypass this by temporarily removing the wcs with `spectrum.wcs = None`, or using the high-level API `WCS` class. However, the first solution is a patchwork solution that doesn't solve the fundamental issue (the default wcs is re-added at some point in my pipeline). I think it is possible to setup the high-level `WCS` for spectral data, but I am unsure how to do it properly. Right now, I am using the `spam` CTYPE of `WCS` to do this, but this resets the `spectrum.spectral_axis` to pixel array in several functions, which is something I would like to avoid.
Is there a workaround that would fix this issue altogether? I think the easiest way is to create the `WCS` correctly, but unfortunately, the [WCS documentation](https://astropy-astrofrog.readthedocs.io/en/latest/wcs/index.html) is not super helpful for the spectral data. As far as I gathered, since my spectral_axis is not linear, I should use the tabulated option, based on this [paper, Greisen et al. (2006), section 6](https://ui.adsabs.harvard.edu/abs/2006A%26A...446..747G/abstract). Is there some convenience function in `specutils` that creates high-level `WCS` given a `spectral_axis`?
Minimal example:
```
import dill # Pickle library
import numpy as np
import specutils as sp
# Mock spectrum
spectrum = sp.Spectrum1D(spectral_axis = np.arange(10)*u.AA,
flux = np.arange(10)*u.dimensionless_unscaled
)
print('Pickleable :' , dill.pickles(spectrum))
# Remove wcs
spectrum.wcs = None
print('Pickleable :' , dill.pickles(spectrum))
# Mock spectrum with high-level API WCS of ctype = 'SPAM'
wcs = WCS(naxis=1)
wcs.wcs.ctype = ['SPAM']
spectrum = sp.Spectrum1D(spectral_axis = np.arange(10)*u.AA,
flux = np.arange(10)*u.dimensionless_unscaled,
wcs = wcs
)
print('Pickleable :' , dill.pickles(spectrum))
```
Contributor guide
Research direction
Start by running the minimal `Spectrum1D` example with `dill` and compare serialization with and without the default `wcs`. Investigate how `Spectrum1D` constructs and stores its WCS and how the high-level WCS affects `spectral_axis`; done means a normal spectrum can be serialized without manually removing `wcs` while retaining the correct spectral axis.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100