Forcefully disallow usage of HadoopFileIO
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 137
Description
### Describe the bug
`HadoopFileIO`, as the name implies, relies a lot on Hadoop dependencies. Hadoop in turn relies a lot on the Java Security Manager, which is [permanently disabled in Java 24](https://inside.java/2024/12/11/quality-heads-up/). This means that starting with Java 24 `HadoopFileIO` can no longer work.
`HadoopFileIO` security model is also different (Java security manager) and quite more complex than the security model of object stores and not that tightly integrated w/ Polaris anyway.
On top, `HadoopFileIO` is not recommended for Polaris production use.
Considering how long it takes Hadoop to catch up w/ Java releases, it is safer to remove all dependencies, mostly test code requirements, to HadoopFileIO.
### To Reproduce
_No response_
### Actual Behavior
_No response_
### Expected Behavior
_No response_
### Additional context
_No response_
### System information
_No response_
Contributor guide
Research direction
Start by searching the repository for HadoopFileIO and Hadoop dependency declarations, especially the test code requirements mentioned in the issue. Confirm which production or test paths still use it and remove those dependencies or usages; done means the build and tests no longer require HadoopFileIO.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100