apache / apache/sedona

GeoPandas: implement distributed sjoin_nearest

Open
#3,182 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.4k
Forks
784
Avg merge
1d 12h
Merged PRs (30d)
58

Description

Part of #2230.

Implement distributed `GeoDataFrame.sjoin_nearest(...)` and top-level `geopandas.sjoin_nearest(...)`.

Requirements:

- Execute through Sedona's distributed K-nearest-neighbor join planning; do not collect geometry rows to the driver, use a Cartesian fallback, or use Python row UDFs.
- Support `inner`, `left`, and `right` joins with GeoPandas-compatible index and suffix behavior.
- Return every equidistant nearest match where GeoPandas returns ties.
- Support `max_distance`, `distance_col`, and `exclusive`.
- Validate CRS compatibility and preserve the active geometry and CRS for each join mode.
- Match GeoPandas behavior for empty and null geometries, empty inputs, duplicate and multi-level indexes, overlapping column names, self-joins, and deterministic result ordering where required.

Add focused tests for the optimized Spark plan as well as direct and GeoPandas parity tests covering join modes, ties, distance limits and output, exclusivity, metadata, index shapes, empty/null inputs, and invalid arguments. Add public API documentation for both entry points.

Contributor guide

Open the contributing guide

Research direction

Start at GeoDataFrame.sjoin_nearest(...) and geopandas.sjoin_nearest(...), then trace Sedona's distributed K-nearest-neighbor join planning. Add focused optimized-plan, direct, and GeoPandas parity tests for the listed join modes, ties, limits, metadata, indexes, empty/null inputs, and invalid arguments. Done means both public entry points match the required GeoPandas behavior without collecting rows or using fallback joins or Python row UDFs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
backend-api-design, data-engineering, distributed-systems, documentation, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.