astronomy-commons / astronomy-commons/lsdb
Add example of pyarrow `filters` to the docs
- Dominant language
- Python
- Stars
- 55
- Forks
- 26
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 8
Description
The docs page for [row filtering](https://docs.lsdb.io/en/latest/tutorials/row_filtering.html) include examples of `Catalog.query(...)` but we are missing instructions on how to use pyarrow filters. E.g.:
```python
gaia_dr3 = lsdb.open_catalog(
"s3://stpubdata/gaia/gaia_dr3/public/hats",
columns=["pmra", "pmdec", "radial_velocity", "parallax"],
filters=[("pm", "<", max_pm), ("parallax", ">", 0.0)],
)
```
We should:
- Highlight that these filters are very efficient for S3 cloud reads (since they are applied server-side).
- Include a pointer to the official [ParquetDataset](https://arrow.apache.org/docs/python/generated/pyarrow.parquet.ParquetDataset.html) docs (see `filters` kwarg) since the syntax for the filters can be unintuitive for users.
Contributor guide
No contributing guide indexed for this repository
Research direction
Open the row filtering docs page and review its existing Catalog.query examples before adding the requested pyarrow filters example. Done means the page explains server-side efficiency for S3 cloud reads and links to the official ParquetDataset documentation for the filters syntax.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100