astropy / astropy/specutils

Serialization of Spectrum1D object (How to make Spectrum1D pickle-able?)

Open
#1,085 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
204
Forks
134
Avg merge
12h 55m
Merged PRs (30d)
1

Description

Hello everyone,

I struggle with serialization (through the `pickle` or `dill` library) of the `sp.Spectrum1D` object. In particular, the default wcs is not pickle-able. I use serialization to both save the data as pickle file and for `multiprocessing` of several functions, so this functionality would be helpful, especially since I actually don't really care about the `sp.Spectrum1D.wcs` attribute.

I know it's possible to bypass this by temporarily removing the wcs with `spectrum.wcs = None`, or using the high-level API `WCS` class. However, the first solution is a patchwork solution that doesn't solve the fundamental issue (the default wcs is re-added at some point in my pipeline). I think it is possible to setup the high-level `WCS` for spectral data, but I am unsure how to do it properly. Right now, I am using the `spam` CTYPE of `WCS` to do this, but this resets the `spectrum.spectral_axis` to pixel array in several functions, which is something I would like to avoid.

Is there a workaround that would fix this issue altogether? I think the easiest way is to create the `WCS` correctly, but unfortunately, the [WCS documentation](https://astropy-astrofrog.readthedocs.io/en/latest/wcs/index.html) is not super helpful for the spectral data. As far as I gathered, since my spectral_axis is not linear, I should use the tabulated option, based on this [paper, Greisen et al. (2006), section 6](https://ui.adsabs.harvard.edu/abs/2006A%26A...446..747G/abstract). Is there some convenience function in `specutils` that creates high-level `WCS` given a `spectral_axis`?

Minimal example:
```
import dill # Pickle library
import numpy as np
import specutils as sp
# Mock spectrum
spectrum = sp.Spectrum1D(spectral_axis = np.arange(10)*u.AA,
flux = np.arange(10)*u.dimensionless_unscaled
)
print('Pickleable :' , dill.pickles(spectrum))

# Remove wcs
spectrum.wcs = None
print('Pickleable :' , dill.pickles(spectrum))

# Mock spectrum with high-level API WCS of ctype = 'SPAM'
wcs = WCS(naxis=1)
wcs.wcs.ctype = ['SPAM']
spectrum = sp.Spectrum1D(spectral_axis = np.arange(10)*u.AA,
flux = np.arange(10)*u.dimensionless_unscaled,
wcs = wcs
)
print('Pickleable :' , dill.pickles(spectrum))

```

Contributor guide

Open the contributing guide

Research direction

Start by running the minimal `Spectrum1D` example with `dill` and compare serialization with and without the default `wcs`. Investigate how `Spectrum1D` constructs and stores its WCS and how the high-level WCS affects `spectral_axis`; done means a normal spectrum can be serialized without manually removing `wcs` while retaining the correct spectral axis.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.