mars-project / mars-project/mars

Conventional schedulers (Slurm/PBS/Loadleveler) compatibility?

Open
#97 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted
Dominant language
Python
Stars
2.7k
Forks
325
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
Most supercomputers in the world use of the few schedulers available, like the ones mentioned at the title. Those usually don't play well with other schedulers.

**Describe the solution you'd like**
To be able to run it directly from a slurm session.

**Describe alternatives you've considered**
In many SPMD environments, one is able to submit one single program to run in a number of compute nodes. So, this program should be able to run as a master in one node, and as a worker in all the other nodes. Something like a front-end to mars.

**Additional context**
Supercomputer schedulers are quite simple in operation. The user submits a job to the batch system, which waits until the amount of resources requested is available. Then, it runs the code in all the processes. It's that simple.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. The issue asks for Mars to run directly from a Slurm session and mentions Slurm, PBS, and Loadleveler compatibility; first define the supported scheduler behavior and integration boundary, then verify that a submitted multi-node job can start the requested master and worker processes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, hpc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.