apache / apache/pinot

Investigate remaining MutableSegmentImpl FIXMEs around upsert consistency and aggregateMetrics

Open
#18,217 1 comment 0 reactions 0 assignees View on GitHub
bug data-integrity java priority: medium real-time upsert
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

### Motivation
`pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java` still has two FIXME blocks:
- around line 187: segment load during consumption can update record location without updating valid doc IDs for a newer late event
- around line 1483: `getOrCreateDocId()` still warns that aggregateMetrics breaks for multi-value dimensions and references closed issue #3867

These comments point at correctness-sensitive realtime behavior and should either be fixed or reconciled with current behavior and tests.

### Scope
- Reproduce or rule out the valid-doc-id inconsistency during concurrent consumption plus segment load.
- Re-evaluate aggregateMetrics behavior for MV dimensions and decide whether code, tests, or comments are stale.
- Add regression coverage for any confirmed bug.
- Remove or update the FIXMEs once behavior is understood.

### Notes
Observed on `upstream/master` on April 15, 2026.

Contributor guide

Open the contributing guide

Research direction

Start in pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java at the FIXME blocks around lines 187 and 1483. Reproduce or rule out the concurrent segment-load inconsistency, then inspect current aggregateMetrics behavior for multi-value dimensions and related tests. Done means confirmed behavior has regression coverage where needed and both FIXMEs are removed or updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.