apache / apache/pinot

Fix SegmentPurger to prevent unintended star-tree modification during purge

Open
#17,821 4 comments 0 reactions 0 assignees View on GitHub
bug minion stale
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

## Problem

When PurgeTask rebuilds a segment (after purging compliance/late-event records), SegmentPurger creates a new segment via SegmentGeneratorConfig which unconditionally picks up starTreeIndexConfigs from the table config, even if enableDynamicStarTreeCreation is set to false. This causes segments to unexpectedly gain star-tree indexes during purge, even when enableDynamicStarTreeCreation is false.

## Proposal

Makes SegmentPurger.initSegmentGeneratorConfig() respect enableDynamicStarTreeCreation — if it's false (default), star-tree configs should not be updated/deleted

Contributor guide

Open the contributing guide

Research direction

Start in SegmentPurger.initSegmentGeneratorConfig() and trace how PurgeTask uses SegmentGeneratorConfig and table configuration when rebuilding a segment. Verify the behavior with enableDynamicStarTreeCreation both false and true; done means purge does not update or delete star-tree configs when the setting is false.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.