Question about "_create_workers"
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 312
- Avg merge
- 1h 2m
- Merged PRs (30d)
- 2
Description
https://github.com/alibaba/ROLL/blob/5ec50dcad87e39bb3b84da7019b44f5f1d5c15ab/roll/distributed/executor/cluster.py#L128C86-L128C95
hi,I am a little confused about the logic here. There may be multiple elements of pgs. Why does the deploy_pg defined here only use the first pg in pgs?
In addition, another question is about the logic of ”num_nodes“. I found that num_nodes will be modified in BaseConfig's __post_init__.
For example, my experimental configuration contains 2 nodes (one h100 per node). I set num_gpus_per_node=1, num_nodes=2 in the configuration file, and device_mapping is "list(range(0,2))". According to the current logic of BaseConfig.__post_init__, num_nodes will be reset to 1.
One is why this logic of resetting num_nodes to 1 occurs? Why can't use the num_nodes=2 I set?
The other is that it feels not very good to reset the user-configured parameters in the code.
Because of this reason, I have been troubleshooting for a long time. I feel that the parameters set by the user should be used as the basis, and the parameters set by the user can be verified instead of resetting (and there is also no prompt when modifying this parameter ). Otherwise it will cause users great confusion.
Thanks for reading.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with roll/distributed/executor/cluster.py at _create_workers and deploy_pg, then inspect BaseConfig.__post_init__ and how pgs, num_nodes, and device_mapping are used. Reproduce the two-node configuration described in the issue; done should include a decided explanation or agreed behavior for worker selection and user-configured num_nodes, with validation or documentation as appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100