org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
i found 0.16.0 Add support minor compaction with segment locking #7547
the supprot contain problem
before 0.16.0 version,we use hadoop-task and kafka-task ingestion data into the same datasource, is success.
but upgrade new version, if one segment is ingestion by hadoop task,the shardSpec will be hashed,
the kafka task ingestion the same segment will exception
org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp
now we must run compact task to solve the exception
Is there any new solution to this problem?
Contributor guide
Research direction
Start by reviewing the 0.16.0 minor-compaction change in issue #7547 and the interaction between Hadoop-task and Kafka-task ingestion for the same datasource. Investigate the hashed shardSpec and the reported segment-allocation exception; done means identifying and documenting a supported solution that avoids the failure without requiring a compaction task.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100