apache / apache/pinot

Do not allow relocating segments for upsert table or realtime to offline task

Open
#8,973 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Currently nothing stops developers from defining segment relocation in their upsert tables using the instanceAssignmentConfigMap. From discussions in the Pinot slack group, it's become apparent that this breaks the upsert functionality for upsert tables (and apparently also the realtime to offline segments task).

We should mark the table configuration as invalid if segment relocation is enabled for an upsert table, or if the realtime to offline task is enabled (cc. @Jackie-Jiang . Can you verify this issue applies to the realtime to offline segments task as well?)

Contributor guide

Open the contributing guide

Research direction

Start by tracing Pinot's table-configuration validation for instanceAssignmentConfigMap and the realtime-to-offline segments task. Confirm how upsert tables and that task are represented, then verify that configurations enabling segment relocation are rejected and covered by validation tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.