Support bootstrap file splits
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3004
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-2687
- Fix version(s):
- 0.11.0
---
## Comments
22/Dec/21 01:30;guoyihua;The implementation is done: [https://github.com/codope/trino/pull/3]
Currently, HBase 1.2.3 used by Hudi is not compatible with Java 11 (required for Trino) when querying the table with bootstrap file format: `Caused by: java.lang.RuntimeException: Unexpected version format: 11.0.13`
[https://gist.github.com/yihua/af275ce96d8f0ecb310e97e0369f7fdc]
;;;
---
01/Nov/23 14:25;codope;Reopened as we can't use `FileStatus`. After [https://github.com/trinodb/trino/pull/17392], Hadoop libs have been removed from the connector. So, the above implementation won't work.;;;
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the implementation linked in the comment and the effects of trinodb/trino#17392, which removed Hadoop libraries from the connector. Inspect how bootstrap file format handling can work without FileStatus; done means querying Hudi tables with bootstrap file splits works on Java 11 without the removed Hadoop dependency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100