apache / apache/dolphinscheduler

[DSIP-80][Scheduler] Fix scheduler duplicate fired

Open
#16,816 0 comments 2 reactions 1 assignee Claimed by @ruanwenjun View on GitHub
DSIP
Dominant language
Java
Stars
14.5k
Forks
5.1k
Avg merge
1d 21h
Merged PRs (30d)
28

Description

### Search before asking

- [X] I had searched in the [DSIP](https://github.com/apache/dolphinscheduler/issues/14102) and found no similar DSIP.

### Motivation

I sometimes find the scheduler might be duplicate triggered.
![image](https://github.com/user-attachments/assets/0b7c6070-f7f3-4ba6-b8f2-4515e0fe546c)

This might happen when `ProcessScheduleTask` has a bad performance or the isolation levels is incorrect or we use incorrect quartz parameters.

The said thing is that we don't have any metrics to monitor whether there exist duplicate fired.

### Design Detail

Add a new table `t_ds_schedule_instance` to record the scheduler fired instance.
```java
CREATE TABLE `t_ds_schedule_instance` (
`id` int NOT NULL AUTO_INCREMENT COMMENT 'key',
`schedule_id` int NOT NULL,
`workflow_instance_id` int NOT NULL,
`expect_fire_time` bigint NOT NULL,
`actual_fire_time` bigint NOT NULL,
`create_time` datetime NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uk_t_ds_schedule_si_eft` (`schedule_id`,`expect_fire_time`)
) ENGINE=InnoDB AUTO_INCREMENT=60 DEFAULT CHARSET=utf8mb3
```
Each record in this table represend a success fired of quartz.

Once the scheduler triggered success we will insert a `ScheduleInstance`, if there exist duplicate record will throw exception from db.

![image](https://github.com/user-attachments/assets/4c9ba1b1-f331-42c0-8ae4-ab48aa25042a)

### Compatibility, Deprecation, and Migration Plan

Compatibility with previous version, since the table is newly created.

### Test Plan

Test by IT.

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.