astronomy-commons / astronomy-commons/lsdb

MOC order for spatial filters could lead to inefficient parquet filters

Open
#959 0 comments 0 reactions 1 assignee Claimed by @smcguire-cmu View on GitHub
enhancement performance
Dominant language
Python
Stars
55
Forks
26
Avg merge
4d 1h
Merged PRs (30d)
8

Description

Currently we determine the MOC order for spatial filters by using the Pixel Tree, since initially the only purpose of the filter MOCs were to filter the pixel tree in the coarse filtering stage. However we now use the filter MOCs to generate pyarrow filters, and this could lead to inefficient filtering. The filtering MOC order could be generated instead from the catalog MOC order or the filter size to make it more efficient.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.