apache / apache/druid

org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp

Open
#9,990 2 comments 0 reactions 0 assignees View on GitHub
Area - Batch Ingestion Bug
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

i found 0.16.0 Add support minor compaction with segment locking #7547
the supprot contain problem

before 0.16.0 version,we use hadoop-task and kafka-task ingestion data into the same datasource, is success.
but upgrade new version, if one segment is ingestion by hadoop task,the shardSpec will be hashed,
the kafka task ingestion the same segment will exception
org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp

now we must run compact task to solve the exception

Is there any new solution to this problem?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the 0.16.0 minor-compaction change in issue #7547 and the interaction between Hadoop-task and Kafka-task ingestion for the same datasource. Investigate the hashed shardSpec and the reported segment-allocation exception; done means identifying and documenting a supported solution that avoids the failure without requiring a compaction task.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.