duckdb / duckdb/duckdb-spatial

Spatial row-group statistics not written on Parquet export; pushdown not available in JOIN context

Open
#807 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
708
Forks
96
Avg merge
1d 21h
Merged PRs (30d)
5

Description

Hi!

We've been exploring the spatial predicate pushdown introduced in 1.5.0, specifically the per-row-group `geo_bbox` statistics and their impact on Parquet scan performance.

## Observations

1. **`geo_bbox` stats are never written on export.** After writing a GeoParquet file via `COPY ... TO '*.parquet'` — including round-tripping a file already recognized as `geometry('ogc:crs84')`, and with an explicit `::GEOMETRY` cast — `parquet_metadata()` shows `geo_bbox = NULL` for all row groups. The geometry column is always written as `BYTE_ARRAY / BLOB` with no spatial statistics. This means the row-group skipping optimization can never fire on files written by DuckDB.

2. **Pushdown doesn't propagate into JOIN operators.** With a constant-geometry `WHERE geometry && ST_GeomFromText(...)` clause, the filter is correctly pushed into `PARQUET_SCAN`. But in a JOIN (`ON a.geom && b.geometry`), the full parquet file is always scanned regardless of how selective the other side is — confirmed by comparing `operator_rows_scanned` and `operator_cardinality` between a 100-polygon join and a single-polygon join (identical scan stats in both cases).

## Questions

- Is writing `geo_bbox` row-group statistics on Parquet export planned? Is there a tracking issue?
- For the JOIN case: is there a path toward using the bounding box of the left-side table to prune row groups on the right side at scan time (e.g., computing the union bbox of the build side before the probe scan begins)?

## Environment

- DuckDB v1.5.2
- `duckdb_spatial` extension
- File validated as valid GeoParquet via `gpq validate`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.