When submitting a compact task that spans multiple days, if one of the days has no data, the entire task will quickly fail

When submitting a compact task that spans multiple days, if one of the days has no data, the entire task will quickly fail?

### Affected Version

0.22.0

### Cluster size

Standalone

### Description

For example, I have a table named TEST_COMPACT, and the structure and data of this table were copied from another table through reindex. This is the spec definition of the reindex task:

``` json
{
"type": "index_parallel",
"spec": {
"ioConfig": {
"type": "index_parallel",
"inputSource": {
"type": "druid",
"dataSource": "TEST_SRC",
"interval": "2023-05-04T00:00:00/2023-05-19T00:00:00"
}
},
"tuningConfig": {
"type": "index_parallel",
"partitionsSpec": {
"type": "dynamic"
},
"maxNumConcurrentSubTasks": 3
},
"dataSchema": {
"timestampSpec": {
"column": "__time",
"format": "millis"
},
"granularitySpec": {
"rollup": true,
"queryGranularity": "minute",
"segmentGranularity": "hour"
},
"dimensionsSpec": {
"dimensions": [
{
"name": "mobile_app_id",
"type": "string"
},
{
"name": "country_id",
"type": "string"
}
]
},
"metricsSpec": [
{
"type": "longSum",
"name": "view_count",
"fieldName": "view_count",
"expression": null
}
],
"dataSource": "TEST_COMPACT"
}
}
}
```
The entire reindex process went smoothly. Then I submitted a compact task, which mainly has two purposes: one is to aggregate minute-level data into hour-level data, and the other is to merge multiple smaller segments.

```json
{
"type":"compact",
"dataSource":"TEST_COMPACT",
"granularitySpec":{
"type":"uniform",
"segmentGranularity":{
"type":"period",
"period":"PT6H"
},
"queryGranularity":"HOUR",
"intervals":[
"2023-05-04T00:00:00/2023-05-19T00:00:00"
]
},
"tuningConfig":{
"type":"index_parallel",
"maxNumConcurrentSubTasks":3,
"forceGuaranteedRollup":"true",
"partitionsSpec":{
"type":"single_dim",
"targetRowsPerSegment":3000000,
"partitionDimension":"mobile_app_id"
}
},
"ioConfig":{
"type":"compact",
"inputSpec":{
"type":"interval",
"interval":"2023-05-04T00:00:00/2023-05-19T00:00:00"
},
"appendToExisting":false
}
}
```

Afterwards, it was discovered that the compact task failed quickly. By checking the overload logs, the following suspected exception logs were found

```text
2023-06-08T12:08:34,491 INFO [qtp617662116-138] org.apache.druid.indexing.overlord.TaskLockbox - Task[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] already present in TaskLock[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]
2023-06-08T12:08:57,532 INFO [qtp617662116-136] org.apache.druid.indexing.overlord.TaskLockbox - Cannot create a new taskLockPosse for request[TimeChunkLockRequest{lockType=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-04T00:00:00.000Z/2023-05-19T00:00:00.000Z, preferredVersion='null', priority=25, revoked=false}] because existing locks[[TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-04T00:00:00.000Z/2023-05-05T12:00:00.000Z, version='2023-06-08T12:08:34.294Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}, TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-06T00:00:00.000Z/2023-05-06T06:00:00.000Z, version='2023-06-08T12:08:34.300Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}, TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-08T00:00:00.000Z/2023-05-08T12:00:00.000Z, version='2023-06-08T12:08:34.309Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}, TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-09T00:00:00.000Z/2023-05-09T12:00:00.000Z, version='2023-06-08T12:08:34.315Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}, TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-10T00:00:00.000Z/2023-05-10T18:00:00.000Z, version='2023-06-08T12:08:34.321Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}, TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-11T00:00:00.000Z/2023-05-11T06:00:00.000Z, version='2023-06-08T12:08:34.329Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}, TaskLockPosse{taskLock=TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-15T00:00:00.000Z/2023-05-19T00:00:00.000Z, version='2023-06-08T12:08:34.336Z', priority=25, revoked=false}, taskIds=[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]}]] have same or higher priorities
2023-06-08T12:08:57,583 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.RemoteTaskRunner - Worker[wukong-v3650-micro.syh.com:8091] wrote FAILED status for task [compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] on [TaskLocation{host='wukong-v3650-micro.syh.com', port=8091, tlsPort=-1}]
2023-06-08T12:08:57,583 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.RemoteTaskRunner - Worker[wukong-v3650-micro.syh.com:8091] completed task[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] with status[FAILED]
2023-06-08T12:08:57,583 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.TaskQueue - Received FAILED status for task: compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z
2023-06-08T12:08:57,583 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.RemoteTaskRunner - Shutdown [compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] because: [notified status change from task]
2023-06-08T12:08:57,583 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.RemoteTaskRunner - Cleaning up task[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] on worker[wukong-v3650-micro.syh.com:8091]
2023-06-08T12:08:57,588 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.TaskLockbox - Removing task[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] from activeTasks
2023-06-08T12:08:57,588 INFO [Curator-PathChildrenCache-3] org.apache.druid.indexing.overlord.TaskLockbox - Removing task[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] from TaskLock[TimeChunkLock{type=EXCLUSIVE, groupId='compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z', dataSource='TEST_COMPACT', interval=2023-05-04T00:00:00.000Z/2023-05-05T12:00:00.000Z, version='2023-06-08T12:08:34.294Z', priority=25, revoked=false}]
```
and some problem logs were found in index.log

```text
2023-06-08T12:08:57,518 INFO [[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]-threading-task-runner-executor-31] org.apache.druid.indexing.common.task.CompactionTask - Generated [1] compaction task specs
2023-06-08T12:08:57,518 INFO [[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]-threading-task-runner-executor-31] org.apache.druid.indexing.common.task.AbstractBatchIndexTask - Using timeChunk lock for perfect rollup
2023-06-08T12:08:57,532 WARN [[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]-threading-task-runner-executor-31] org.apache.druid.indexing.common.task.CompactionTask - indexSpec is not ready: [{
"id" : "compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z",
"groupId" : "compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z",
"availabilityGroup" : "compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z",
"appenderatorTrackingTaskId" : "compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z",
2023-06-08T12:08:57,532 INFO [[compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]-threading-task-runner-executor-31] org.apache.druid.indexing.common.task.CompactionTask - Ran [1] specs, [0] succeeded, [1] failed
2023-06-08T12:08:57,532 ERROR [threading-task-runner-executor-31] org.apache.druid.segment.realtime.appenderator.UnifiedIndexerAppenderatorsManager - Could not find datasource bundle for [TEST_COMPACT], task [compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z]
2023-06-08T12:08:57,558 INFO [threading-task-runner-executor-31] org.apache.druid.indexing.overlord.ThreadingTaskRunner - Removed task directory: var/druid/task/compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z
2023-06-08T12:08:57,583 INFO [WorkerTaskManager-NoticeHandler] org.apache.druid.indexing.worker.WorkerTaskManager - Task [compact_TEST_COMPACT_nadfdjkg_2023-06-08T12:08:34.278Z] completed with status [FAILED].
```
Later, I saw the following description in the "compact" section of the official documentation:

> Compaction tasks exit without doing anything and issue a failure status code in either of the following cases:
> - If the interval you specify has no data segments loaded.
> - If the interval you specify is empty.

So I discovered by observing the Druid web management page that there were indeed some days without data in the TEST_COMPACT datasource between 2023-05-04T00:00:00 and 2023-05-19T00:00:00. Therefore, I tried resubmitting a merge task with data for each day within the interval, and the merge task was able to run smoothly. Strangely enough, this doesn't seem to be the scenario described in the documentation.

I don't quite understand why the Druid compact task fails directly when encountering a time interval without data, instead of prompting in the compact result report that there is no data to compact. Also, when I try to perform a similar compact action through reindex, it works perfectly fine.

Start with CompactionTask and AbstractBatchIndexTask, using the reported compact interval containing empty days to reproduce the failure. Compare this path with the reindex task behavior and inspect the UnifiedIndexerAppenderatorsManager error; done means compaction handles empty subintervals consistently and reports the result without failing the whole task.

Written by the indexing model from the issue text.