apache / apache/druid

Support Apache Ozone as Deep Storage

Open
#20,224 1 comment 1 reaction 0 assignees View on GitHub
Feature/Change Description
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Description
We would like to request support for **Apache Ozone** as a deep storage option in Apache Druid.
Apache Ozone provides an S3-compatible object store and is designed to integrate with the Hadoop ecosystem. Since the Hadoop version used by the Druid `druid-hdfs-storage` extension is compatible with the Hadoop libraries required by Apache Ozone, we believe Ozone could potentially be supported without introducing a completely separate storage implementation.

We would therefore like to ask whether support for Apache Ozone could be added either:
1. **To the existing `druid-hdfs-storage` extension**, leveraging the Hadoop/Ozone filesystem integration; or
2. **Through a dedicated Druid Ozone storage extension**, if integrating it into `druid-hdfs-storage` is not considered appropriate.

### Motivation
We are using Apache Ozone as the object storage layer of our Big Data platform and would like to use it as Druid's deep storage as well.
Apache Ozone provides a Hadoop-compatible filesystem interface, allowing applications to access Ozone through the Hadoop `FileSystem` API. This could potentially allow Druid to access Ozone directly through its existing Hadoop-based deep storage implementation.

Although Apache Ozone provides an S3-compatible API, relying on its S3 Gateway is not ideal for our use case, as it introduces an additional access layer and associated request overhead. Direct access through Ozone's native Hadoop `FileSystem` integration would avoid this extra layer and provide lower-latency access to deep storage.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing druid-hdfs-storage extension and its Hadoop FileSystem integration. Determine whether Apache Ozone can be supported there or requires a dedicated Druid Ozone storage extension; done means Druid can use Ozone directly as deep storage without relying on the S3 Gateway.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cloud, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.