Add Table statistics / Row estimation for ParquetTable
Open
dsl
improvement
io
java
P3
parquet
sql
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Table statistics and Row estimation are used by the SQL implementation to optimize joins so this information may help improve BeamSQL performance on Parquet datasets.
Imported from Jira [BEAM-12134](https://issues.apache.org/jira/browse/BEAM-12134). Original Jira may contain additional context.
Reported by: iemejia.
Contributor guide
Research direction
Start by locating ParquetTable and the Beam SQL code that consumes table statistics and row estimates for join optimization. Trace how Parquet datasets currently provide this information, then verify that both statistics and row estimation are available to the SQL implementation and improve optimization for Parquet data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100