astronomy-commons / astronomy-commons/lsdb

Lazy filtering of empty partitions after spatial filters

Open
#199 0 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
Python
Stars
55
Forks
26
Avg merge
4d 1h
Merged PRs (30d)
8

Description

When we apply a filter (cone, polygonal or box), we may carry empty partitions in the catalog for the remainder of the computations but we can reduce its likelihood.

Here is a small example of applying a cone search to a catalog. It doesn’t make sense to keep the entire patch of sky covered by partitions 1 to 4 because they have large regions where they won’t overlap with the filter. By increasing the order of partitions 1 to 4 we can remove the regions of each partition around the filter that we won’t need.

Screenshot 2024-03-01 at 5 13 50 PM

Contributor guide

No contributing guide indexed for this repository

Research direction

No files or tests are named. Start by tracing how cone, polygonal, and box filters select catalog partitions, then determine how partition order can reduce regions that do not overlap the filter. Done means spatial filtering avoids carrying unnecessary empty partition regions through later computations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.