apache / apache/dolphinscheduler
[DSIP-69] Fix master dispatch task timeout might cause task duplicate running in worker
- Dominant language
- Java
- Stars
- 14.5k
- Forks
- 5.1k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 28
Description
### Search before asking
- [X] I had searched in the [DSIP](https://github.com/apache/dolphinscheduler/issues/14102) and found no similar DSIP.
### Motivation
Right now, there exist some case might cause the task duplicated dispatched.
e.g.
The master dispatch task a to worker A first, but receive a timeout response, this might happen when the worker rpc is busy, then master will select a new worker B and retry the dispatch.
Then there might exist two situations:
1. The task has been received by worker A, then take will duplicate exist in worker A and worker B, both the two worker will execute the task, a worser case is the task might duplicated in more worker.
2. The task hasn't been received by worker A, then task will not duplicate executed.
The first situation is not accepted.
### Design Detail
In order to solve this, we should change the dispatch logic.
### Compatibility, Deprecation, and Migration Plan
_No response_
### Test Plan
_No response_
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Assessment
This issue has not been assessed yet.