apache / apache/datafusion-ballista
Scheduler will lost the registered executor when restart it in the `push` mode
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 320
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 66
Description
**Describe the bug**
When i restart the schedule, the schedule lost all the information of registered executor
**To Reproduce**
Start a scheduler with below config:
```
scheduler_policy="PushStaged"
```
Start a executor with below config:
```
scheduler_port=50050
scheduler_host="localhost"
# PushStaged or PullStaged
task_scheduling_policy="PushStaged"
```
then kill the scheduler and restart the scheduler using the same config.
And the scheduler will lost all registered executor in the memory.
**Expected behavior**
We should recover this data in memory after the scheduler restart.
Solution:
heartbeat with the registered information for the executor
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
Start by reproducing the restart scenario with scheduler_policy="PushStaged" and the provided executor configuration. Trace how registered executors are held and how heartbeat messages are handled; done means restarting the scheduler restores the executor registration in memory and the executor remains usable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100