mars-project / mars-project/mars

Make `run_subtask` API support batch operation

Open
#3,089 0 comments 0 reactions 0 assignees View on GitHub
mod: scheduling service type: enhancement
Dominant language
Python
Stars
2.7k
Forks
325
PR merge metrics
No merged PRs in 30d

Description

In current implementation `SubtaskExecutionActor.run_subtask` keeps the whole lifecycle of a subtask. This disables the possibility to submit subtasks in batches, which is a major time cost of supervisors. As all calls of `SchedulingAPI.finish_subtasks` own `SubtaskResult` already, it is possible to make `run_subtask` to start subtasks only and stops tracking subtask execution.

Contributor guide

Open the contributing guide

Research direction

Start by tracing SubtaskExecutionActor.run_subtask and the calls to SchedulingAPI.finish_subtasks to understand the current lifecycle and result ownership. Done means run_subtask can start subtasks without tracking their execution, while batch submission and finish_subtasks continue to handle SubtaskResult values correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.