Support multiple deep stores in Pinot Controller with ReadOnly and ReadWrite mode
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
Current pinot only supports one deep store, it works in most of the cases, but makes it hard to migrate data from one deep store to another.
The goal here is to allow multiple deep stores in Pinot controllers and allow config R/W permission per deep store basis.
E.g. we can config the old deep store as ReadOnly and the new deep store as ReadWrite. Then migrating segments from old deep store to new deep store and update the download URL.
Contributor guide
Research direction
Start with the Pinot Controller's current single deep-store configuration and the segment migration and download-URL paths described in the issue. Define how multiple stores select ReadOnly or ReadWrite access, then verify that segments can migrate from the old store to the new store and use the updated download URL. No specific file or test is named, so locating the relevant controller code is part of the work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100