bcgov / bcgov/FIT_changedetector
normalize geoparquet geometry column names
- Dominant language
- Python
- Stars
- 2
- Forks
- 1
- Avg merge
- 9h 49m
- Merged PRs (30d)
- 41
Description
Geoparquet is special because it is read with pyarrow not pyogrio, so the geometry name is not standardized.
Comparing GeoParquet with a.geometry.name="geometry", b.geometry.name="geom" fails:
File ".../changedetector.py", line 828, in gdf_diff
m_geometries = m_geometries[fields_b_src].reset_index(drop=True)
KeyError: "['geom'] not in index"
Crashes even for a diff with no geometry changes at all (just an attribute edit) — because gdf_diff() unconditionally builds the (possibly-empty) MODIFIED_GEOM output for any spatial comparison.
Root cause: in _validate_and_prepare_diff_inputs, df_a_src/df_b_src are copied before each source's geometry column is renamed to the canonical "geometry" — so they deliberately keep their own original column names for rebuilding full-schema output. But gdf_diff()'s m_geometries reconstruction merges df_b_src (still has "geom") against geometry data that's already been standardized to "geometry", then tries to re-select df_b_src's original column list — which still says "geom", a column that no longer exists in the merged frame.
Contributor guide
Research direction
Start in changedetector.py at _validate_and_prepare_diff_inputs and gdf_diff(), following how each source geometry column is renamed and how m_geometries is rebuilt. Reproduce the geometry/geom comparison with an attribute-only edit, then verify that spatial diffs complete without a missing-column error and preserve the expected canonical geometry output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100