duckdb / duckdb/duckdb-spatial
Spatial row-group statistics not written on Parquet export; pushdown not available in JOIN context
- Dominant language
- C
- Stars
- 708
- Forks
- 96
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 5
Description
Hi!
We've been exploring the spatial predicate pushdown introduced in 1.5.0, specifically the per-row-group `geo_bbox` statistics and their impact on Parquet scan performance.
## Observations
1. **`geo_bbox` stats are never written on export.** After writing a GeoParquet file via `COPY ... TO '*.parquet'` — including round-tripping a file already recognized as `geometry('ogc:crs84')`, and with an explicit `::GEOMETRY` cast — `parquet_metadata()` shows `geo_bbox = NULL` for all row groups. The geometry column is always written as `BYTE_ARRAY / BLOB` with no spatial statistics. This means the row-group skipping optimization can never fire on files written by DuckDB.
2. **Pushdown doesn't propagate into JOIN operators.** With a constant-geometry `WHERE geometry && ST_GeomFromText(...)` clause, the filter is correctly pushed into `PARQUET_SCAN`. But in a JOIN (`ON a.geom && b.geometry`), the full parquet file is always scanned regardless of how selective the other side is — confirmed by comparing `operator_rows_scanned` and `operator_cardinality` between a 100-polygon join and a single-polygon join (identical scan stats in both cases).
## Questions
- Is writing `geo_bbox` row-group statistics on Parquet export planned? Is there a tracking issue?
- For the JOIN case: is there a path toward using the bounding box of the left-side table to prune row groups on the right side at scan time (e.g., computing the union bbox of the build side before the probe scan begins)?
## Environment
- DuckDB v1.5.2
- `duckdb_spatial` extension
- File validated as valid GeoParquet via `gpq validate`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.