apache / apache/pinot

[Feature Request] Local Data Directory In Other File Systems [like Hadoop]

Open
#10,911 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Need support for temporary directory to point to any other file system other than local file system
If the disk is full , servers will go down as our machine's monitoring process.
Need Alternatives For configurations like,
controller.local.temp.dir
pinot.server.instance.dataDir
pinot.server.instance.segmentTarDir

Contributor guide

Open the contributing guide

Research direction

Start by tracing the configuration entry points named in the issue: controller.local.temp.dir, pinot.server.instance.dataDir, and pinot.server.instance.segmentTarDir. Determine the supported storage abstraction and requirements for pointing these directories at non-local file systems such as Hadoop; done should include a defined configuration approach that avoids relying on local disk capacity.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, java
Domain
backend, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.