Add col stats to streaming writes to metadata table.
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Task Description
**What needs to be done:**
Migrate col stats generation from batch generation to streaming writes to metadata table.
**Why this task is needed:**
Efficient col stats generation
### Task Type
Code improvement/refactoring
### Related Issues
**Parent feature issue:** (if applicable )
https://github.com/apache/hudi/issues/17491
**Related issues:**
NOTE: Use `Relationships` button to add parent/blocking issues after issue is created.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading parent feature issue #17491 and tracing the existing batch column-statistics generation and streaming writes to the metadata table. The issue names no files or tests, so identify the relevant Hudi modules and verification coverage before making changes. Done means column statistics are generated during streaming writes rather than batch generation, with existing behavior preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100