bcgov / bcgov/FIT_changedetector

normalize geoparquet geometry column names

Closed
#121 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2
Forks
1
Avg merge
9h 49m
Merged PRs (30d)
41

Description

Geoparquet is special because it is read with pyarrow not pyogrio, so the geometry name is not standardized.

Comparing GeoParquet with a.geometry.name="geometry", b.geometry.name="geom" fails:

File ".../changedetector.py", line 828, in gdf_diff
m_geometries = m_geometries[fields_b_src].reset_index(drop=True)
KeyError: "['geom'] not in index"

Crashes even for a diff with no geometry changes at all (just an attribute edit) — because gdf_diff() unconditionally builds the (possibly-empty) MODIFIED_GEOM output for any spatial comparison.

Root cause: in _validate_and_prepare_diff_inputs, df_a_src/df_b_src are copied before each source's geometry column is renamed to the canonical "geometry" — so they deliberately keep their own original column names for rebuilding full-schema output. But gdf_diff()'s m_geometries reconstruction merges df_b_src (still has "geom") against geometry data that's already been standardized to "geometry", then tries to re-select df_b_src's original column list — which still says "geom", a column that no longer exists in the merged frame.

Contributor guide

Open the contributing guide

Research direction

Start in changedetector.py at _validate_and_prepare_diff_inputs and gdf_diff(), following how each source geometry column is renamed and how m_geometries is rebuilt. Reproduce the geometry/geom comparison with an attribute-only edit, then verify that spatial diffs complete without a missing-column error and preserve the expected canonical geometry output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.