Instants file read write should replace byte array allocation with input output stream.
Open
from-jira
priority:high
type:bug
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
For relevant code path on archival and commits where we need to read write instant files, underlying code can use byte array and is not robust for handling big instant files.
One example is org.apache.hudi.common.table.log.HoodieLogFormatWriter#appendBlocks. When we ask for byte array 2 GB and larger JVM will simply reject the request and code will fail.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8822
- Type: Bug
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.