apache / apache/datafusion-ballista

Scheduler will lost the registered executor when restart it in the `push` mode

Open
#90 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
2.1k
Forks
320
Avg merge
1d 22h
Merged PRs (30d)
66

Description

**Describe the bug**
When i restart the schedule, the schedule lost all the information of registered executor

**To Reproduce**
Start a scheduler with below config:
```
scheduler_policy="PushStaged"
```
Start a executor with below config:
```
scheduler_port=50050
scheduler_host="localhost"
# PushStaged or PullStaged
task_scheduling_policy="PushStaged"
```

then kill the scheduler and restart the scheduler using the same config.

And the scheduler will lost all registered executor in the memory.

**Expected behavior**
We should recover this data in memory after the scheduler restart.

Solution:
heartbeat with the registered information for the executor

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the restart scenario with scheduler_policy="PushStaged" and the provided executor configuration. Trace how registered executors are held and how heartbeat messages are handled; done means restarting the scheduler restores the executor registration in memory and the executor remains usable.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.