apache / apache/opendal

new feature: Use hdfs jni directly instead of using `libhdfs`

Open
#5,119 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
5.4k
Forks
825
Avg merge
1d 14m
Merged PRs (30d)
127

Description

### Feature Description

`libhdfs` needs to call Java via `Rust -> C -> Java`, which introduces additional overhead. It might be more efficient to call JNI directly, bypassing the dependency on `libhdfs`.

### Problem and Solution

Implement hdfs via `jni` directly without `libhdfs`.

### Additional Context

It primarily seems like rewriting `libhdfs` in Rust.

### Are you willing to contribute to the development of this feature?

- [ ] Yes, I am willing to contribute to the development of this feature.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing libhdfs integration and the JNI interface involved in calling Java from Rust. Define the supported HDFS operations and compatibility requirements before attempting the proposed rewrite; the issue names no files or tests, so the completion criteria need to be established first.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, rust
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.