alibaba / alibaba/EasyParallelLibrary

epl单机单卡和单机多卡训练step如何理解

Open
#30 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
272
Forks
50
PR merge metrics
No merged PRs in 30d

Description

单机单卡:
启动命令:TF_CONFIG='{"cluster":{"worker":["127.0.0.1:49119"]},"task":{"type":"worker","index":0}}' CUDA_VISIBLE_DEVICES=0 bash ./scripts/train_dp.sh
![image](https://github.com/alibaba/EasyParallelLibrary/assets/28698695/bc38cc24-48b4-452c-8993-546847954fb4)

单机双卡:
启动命令:TF_CONFIG='{"cluster":{"worker":["127.0.0.1:49119"]},"task":{"type":"worker","index":0}}' CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/train_dp.sh
![1693045873752](https://github.com/alibaba/EasyParallelLibrary/assets/28698695/20739a66-8667-46f8-bc5d-1fd5993b0ba5)

代码修改了一下:去掉了last_step限制,数据集repeat=10,将txt改为py,可执行。
[resnet_dp.txt](https://github.com/alibaba/EasyParallelLibrary/files/12671704/resnet_dp.txt)

想请教下,这个如何理解呢?每个卡分别跑了10step?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with scripts/train_dp.sh and the attached resnet_dp.txt/resnet_dp.py example, then compare the single-GPU and two-GPU commands with their reported steps. Document how dataset repeat, worker configuration, and per-device execution affect the step count; done means the issue has a clear explanation supported by the observed runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.