apache / apache/polaris

[FEATURE REQUEST] Add basic HDFS storage option for catalogs

Open
#85 11 comments 5 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 17h
Merged PRs (30d)
137

Description

**Is your feature request related to a problem? Please describe.**
Currently it appears that the storage options are geared towards cloud providers. To support companies running on premise I would like to request HDFS support.

**Describe the solution you'd like**
Catalog R/W support HDFS as a storage option.

For the first implementation we would like something basic:

- The service is able to R/W against a HDFS cluster without authentication/authorization required.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how the catalog currently implements storage options and identify the entry point for catalog reads and writes. Assess the minimal HDFS integration needed for an unauthenticated, on-premise HDFS cluster, then verify that catalog read and write operations work against such a cluster.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.