apache / apache/gravitino

[Improvement] Make Gravitino filesystem support both Hadoop 2 and Hadoop3

Open
#2,827 0 comments 0 reactions 0 assignees View on GitHub
help wanted improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 15h
Merged PRs (30d)
315

Description

### What would you like to be improved?

Current Gravitino virtual filesystem (filesystem-hadoop3) module only builds against Hadoop 3, doesn't have a shim layer to support multiple Hadoop versions. it is better to filesystem-hadoop3 module to support multiple Hadoop versions.

### How should we improve?

I think we can refer to several projects to build a shim layer to support multiple Hadoop versions if necessary.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the filesystem-hadoop3 module and its current Hadoop 3 build and API usage. Compare the module’s version-specific dependencies and entry points with the requirements for Hadoop 2 support, then review shim-layer approaches in the referenced projects. Done means the Gravitino virtual filesystem builds and works against both Hadoop 2 and Hadoop 3.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.