apache / apache/pinot

Organize segments into time based folders when using S3 as deep storage

Open
#5,609 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

Currently all the segments which are uploaded to S3 are being stored in a single folder,
it would be nice to group them into time window based folders.

Contributor guide

Open the contributing guide

Research direction

The issue names no files or tests. Start by locating the S3 deep-storage segment upload path and tracing how segment locations are formed. Define the time-window folder behavior and verify that uploaded segments are grouped into the agreed time-based folders.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.