apache / apache/pinot

Support multiple deep stores in Pinot Controller with ReadOnly and ReadWrite mode

Open
#7,302 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Current pinot only supports one deep store, it works in most of the cases, but makes it hard to migrate data from one deep store to another.

The goal here is to allow multiple deep stores in Pinot controllers and allow config R/W permission per deep store basis.

E.g. we can config the old deep store as ReadOnly and the new deep store as ReadWrite. Then migrating segments from old deep store to new deep store and update the download URL.

Contributor guide

Open the contributing guide

Research direction

Start with the Pinot Controller's current single deep-store configuration and the segment migration and download-URL paths described in the issue. Define how multiple stores select ReadOnly or ReadWrite access, then verify that segments can migrate from the old store to the new store and use the updated download URL. No specific file or test is named, so locating the relevant controller code is part of the work.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.