GeoPandas: implement distributed sjoin_nearest
- Dominant language
- Java
- Stars
- 2.4k
- Forks
- 784
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 58
Description
Part of #2230.
Implement distributed `GeoDataFrame.sjoin_nearest(...)` and top-level `geopandas.sjoin_nearest(...)`.
Requirements:
- Execute through Sedona's distributed K-nearest-neighbor join planning; do not collect geometry rows to the driver, use a Cartesian fallback, or use Python row UDFs.
- Support `inner`, `left`, and `right` joins with GeoPandas-compatible index and suffix behavior.
- Return every equidistant nearest match where GeoPandas returns ties.
- Support `max_distance`, `distance_col`, and `exclusive`.
- Validate CRS compatibility and preserve the active geometry and CRS for each join mode.
- Match GeoPandas behavior for empty and null geometries, empty inputs, duplicate and multi-level indexes, overlapping column names, self-joins, and deterministic result ordering where required.
Add focused tests for the optimized Spark plan as well as direct and GeoPandas parity tests covering join modes, ties, distance limits and output, exclusivity, metadata, index shapes, empty/null inputs, and invalid arguments. Add public API documentation for both entry points.
Contributor guide
Research direction
Start at GeoDataFrame.sjoin_nearest(...) and geopandas.sjoin_nearest(...), then trace Sedona's distributed K-nearest-neighbor join planning. Add focused optimized-plan, direct, and GeoPandas parity tests for the listed join modes, ties, limits, metadata, indexes, empty/null inputs, and invalid arguments. Done means both public entry points match the required GeoPandas behavior without collecting rows or using fallback joins or Python row UDFs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- backend-api-design, data-engineering, distributed-systems, documentation, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100