Assess handling of large cubes
- Dominant language
- Python
- Stars
- 204
- Forks
- 134
- Avg merge
- 12h 55m
- Merged PRs (30d)
- 1
Description
This is probably a task that should wait until some of the other `spectral-cube` transition work is done. Essentially, `spectral-cube` takes some care in handling large cubes, in terms of avoiding copying data and providing hints on how to use memory mapping (from fits files) to handle cubes that are too big to fit in memory. It's possible that `specutils` handling of large data is already fine, but it would be prudent to do a side-by-side test with `spectral-cube` of a large dataset and see if some effort to improve large dataset handling/port over some strategies from `spectral-cube` would be prudent.
Contributor guide
Research direction
Start by reviewing specutils and spectral-cube handling of a large dataset side by side. Check whether specutils avoids unnecessary copies and supports memory mapping for FITS data that exceeds available memory. Done means documenting the comparison and determining whether specific handling strategies should be ported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100