Make sure HoodieTableMetadata API allows to read both in-memory and on-cluster
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 112
Description
Currently, most of the HoodieTableMetadata APIs (getColumnStats, getBloomFilters, etc) provide only for loading the data from MT in-memory which shifts the burden on the caller to break their requests in chunks to make sure these fit in memory: for ex, trying to read Bloom Filters from MT, Bloom Index (caller) have to make sure we're not reading more than 256 filters at a time to limit its memory footprint.
Instead HoodieTableMetadata API has to be
# Rebased to rely on HoodieData
# Provide levers to the caller whether MT should be read in-memory or on-cluster
#
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5556
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-1292
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.