apache / apache/pinot

Create an API for cloning a table

Open
#11,781 4 comments 0 reactions 0 assignees View on GitHub
feature stale
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

Per https://docs.pinot.apache.org/users/tutorials/schema-evolution, most schema changes aside from adding a column are not supported. In order to be able to rollback to a previous schema, we want to be able to clone a table using the same underlying servers so we can potentially switch back to the old copy. Ideally this only clones the ZK metadata so we don't have to re-ingest all the segments again.

Cloning a dataset can also be useful to move it to a new tenant for performance testing, etc that wouldn't impact production query traffic to the existing table.

cc @Jackie-Jiang @jadami10

Contributor guide

Open the contributing guide

Research direction

Start with the schema-evolution tutorial linked in the issue and inspect the existing table-management API entry points. Define how cloning only ZK metadata can create a copy on the same servers, support rollback to the previous table, and optionally move the clone to another tenant without re-ingesting segments.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.