microsoft / microsoft/pai

Support for different hardware configurations for different task roles of one distributed job.

Open
#5,808 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
2.7k
Forks
554
Avg merge
20h 42m
Merged PRs (30d)
14

Description

What would you like to be added:
Support for different hardware configurations for different task roles of one distributed job.

Why is this needed:
For complex learning tasks, the programs that need to run on each computer are very different, and the requirements for CPU /GPU and RAM /GPU memory are also different. At the same time, these computers need to communicate with each other to enable joint training. For example, in reinforcement learning, the entire reinforcement learning algorithm consists of different modules. The actor uses the GPU to generate data, the learner uses the GPU to train data, the environment and MCTS use CPU to generate data in parallel, and these modules involve complex data communication.

Without this feature, how does the current module work:
Reinforcement learning tasks cannot be performed jointly by multiple computers.

Components that may involve changes:
Job protocol and related.

Downgrade vc to taskrole:
image

Allows each taskrole to have a different skutype:
image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the job protocol and the existing taskrole and skutype handling mentioned in the issue. Determine how one distributed job currently represents hardware requirements, then define support for distinct configurations per task role and verify that reinforcement-learning-style roles can communicate and schedule correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kubernetes
Domain
distributed-systems, infrastructure, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.