apache / apache/druid

Overwriting tasks doesn't guarantee atomic update of segments across time chunks

Open
#9,973 2 comments 0 reactions 0 assignees View on GitHub
Area - Batch Ingestion Improvement
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

All versions

### Description

The timeline currently doesn't guarantee that all segments of an overwriting task will replace all old segments atomically. This can lead to inconsistency in the query result; a query can read mixed versions of segments across time chunks

Contributor guide

Open the contributing guide

Research direction

The issue concerns timeline overwrites and inconsistent reads of segments across time chunks, but names no files, tests, or entry points. Start by locating the timeline overwrite and segment-read paths, then identify how an overwrite can expose mixed versions; done means queries no longer observe mixed segment versions during an overwrite.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.