Overwriting tasks doesn't guarantee atomic update of segments across time chunks
Open
Area - Batch Ingestion
Improvement
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Affected Version
All versions
### Description
The timeline currently doesn't guarantee that all segments of an overwriting task will replace all old segments atomically. This can lead to inconsistency in the query result; a query can read mixed versions of segments across time chunks
Contributor guide
Research direction
The issue concerns timeline overwrites and inconsistent reads of segments across time chunks, but names no files, tests, or entry points. Start by locating the timeline overwrite and segment-read paths, then identify how an overwrite can expose mixed versions; done means queries no longer observe mixed segment versions during an overwrite.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100