apache / apache/pinot

Reload Table API to reload table metadata / configuration

Open
#10,916 0 comments 2 reactions 0 assignees View on GitHub
design-review
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Currently Pinot doesn’t have a way to reload tables and its related metadata when needed. This requires us to restart the Pinot servers for it to fetch the updated metadata. Few examples include:
1. update a table’s upsert mode from “full-upsert” to “partial-upsert” - #10833
2. marking a table as dimTable requires server restarts - #10668

This [write-up](https://docs.google.com/document/d/1G7qMeuumPLeoTY1wX6dqgVphAdEGRVA1PbpPTL6--AE/edit?usp=sharing) proposes a solution to solve this. Wanted to get the community's input on this before proceeding with the implementation.

cc @Jackie-Jiang @tibrewalpratik17 for review.

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked design write-up and the related examples in issues #10833 and #10668. The issue is currently asking for community input before implementation; done would be a reviewed solution that reloads table metadata and configuration without restarting Pinot servers.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.