astronomy-commons / astronomy-commons/lsdb
Explore supporting dedicated MOC metadata file
- Dominant language
- Python
- Stars
- 55
- Forks
- 26
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 8
Description
Currently we load the full skymap histogram to load the coverage MOC in the lazy stage of `open_catalog`. For large, non-sparse catalog, this can result in loading fairly large files in the lazy part of the first workflow, since we load the entire histogram and drop the density part, keeping the coverage. We could store a MOC file that only keeps the coverage which would be smaller to load, making the lazy open faster.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the open_catalog lazy stage and how the skymap histogram supplies the coverage MOC. Measure the current loading behavior for large, non-sparse catalogs, then evaluate whether a dedicated coverage-only MOC file can be stored and consumed. Done means the approach and its effect on lazy opening are documented or implemented with validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100