alibaba / alibaba/ROLL

Question about "_create_workers"

Open
#75 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.4k
Forks
312
Avg merge
1h 2m
Merged PRs (30d)
2

Description

https://github.com/alibaba/ROLL/blob/5ec50dcad87e39bb3b84da7019b44f5f1d5c15ab/roll/distributed/executor/cluster.py#L128C86-L128C95

hi,I am a little confused about the logic here. There may be multiple elements of pgs. Why does the deploy_pg defined here only use the first pg in pgs?

In addition, another question is about the logic of ”num_nodes“. I found that num_nodes will be modified in BaseConfig's __post_init__.
For example, my experimental configuration contains 2 nodes (one h100 per node). I set num_gpus_per_node=1, num_nodes=2 in the configuration file, and device_mapping is "list(range(0,2))". According to the current logic of BaseConfig.__post_init__, num_nodes will be reset to 1.
One is why this logic of resetting num_nodes to 1 occurs? Why can't use the num_nodes=2 I set?
The other is that it feels not very good to reset the user-configured parameters in the code.
Because of this reason, I have been troubleshooting for a long time. I feel that the parameters set by the user should be used as the basis, and the parameters set by the user can be verified instead of resetting (and there is also no prompt when modifying this parameter ). Otherwise it will cause users great confusion.

Thanks for reading.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with roll/distributed/executor/cluster.py at _create_workers and deploy_pg, then inspect BaseConfig.__post_init__ and how pgs, num_nodes, and device_mapping are used. Reproduce the two-node configuration described in the issue; done should include a decided explanation or agreed behavior for worker selection and user-configured num_nodes, with validation or documentation as appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.