apache / apache/parquet-site

Add more granular Geometry / Geography support to the implementation status page

Open Beginner friendly
#158 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
HTML
Stars
17
Forks
55
Avg merge
1h 25m
Merged PRs (30d)
1

Description

This ticket tries to capture the disucsion with @steveloughran, @csringhofer, myself and others on https://github.com/apache/parquet-site/pull/156#pullrequestreview-3772364529

It's been pointed out to me that the coverage matrix doesn't cover statistics/geometry bounding, without which predicate pushdown doesn't work: every rowgroup with the column needs scanning.

The core point as I understand it is that there are several features that must be implemented in software libraries to realize the full benefits of the new Geometry and Geography types in Parquet. Specifically mentioned were

  • Logical type annotation (to know what columns hold Geometry and Geography types) <-- this is what the page currently reflects
  • Statistics implementation (e.g. the bounding boxes, and potentially different algorithms to compute them)
  • Query engine implementation (e.g. using the bounding box statistics to prune parquet files at query time)

There are probably more

Suggestions

One the idea is to add more specific detail on https://parquet.apache.org/docs/file-format/implementationstatus/ .

Image

Perhaps it would be appropriate to add a specific line for the geography/geometry statistics, for example

In addition to making the current implementation status more clear, red X's on the page seems to have the effect of pressuring additional ecosystem adoption.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the implementation status page at parquet.apache.org/docs/file-format/implementationstatus/ and review the linked parquet-site pull request discussion. Update the Geometry/Geography coverage details to distinguish logical type, statistics, and query-engine support. Done means the page clearly represents the more granular implementation status.

Written by the indexing model from the issue text.

Assessment

Tech stack
html
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.