tracking issue: Bump arrow-go / geoarrow-go versions to add logic type resolution for geo extension types
- Dominant language
- Go
- Stars
- 463
- Forks
- 232
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 118
Description
_Creating this mainly as an issue to track progress / dependencies in external repos._
Currently in iceberg-go geo data can be read / written, but the parquet logical type labeling is wrong leading to issues like this for query engines: https://github.com/duckdb/duckdb-iceberg/issues/1175 The iceberg schema shows geometry and the physical byte array type is correct but the parquet logical type isn't resolving the logical type mapping from the geo extension type.
I have added logic in arrow-go to fix this. It created a new interface for resolving logical type <-> physical type for an arrow extension type.
- [x] https://github.com/apache/arrow-go/pull/1028
- [x] https://github.com/apache/arrow-go/pull/969
- [x] https://github.com/apache/arrow-go/pull/960
And then in geoarrow-go I have implemented the necessary associated read/write interfaces:
- https://github.com/geoarrow/geoarrow-go/pull/5
- https://github.com/geoarrow/geoarrow-go/pull/6
Outstanding tasks:
- Once arrow-go has its version bumped, I will create a PR to bump the arrow-go version in geoarrow-go and then the arrow-go version in iceberg-go will need to be bumped.
Contributor guide
Research direction
Start by checking the completed arrow-go and geoarrow-go pull requests and the dependency declarations in those repositories. After the arrow-go version is available, update the geoarrow-go and iceberg-go versions in sequence, then verify that geo data preserves the correct Parquet logical type labeling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- build-system, data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100