分布式训练worker问题
Open
- Dominant language
- C++
- Stars
- 2.9k
- Forks
- 553
- PR merge metrics
- No merged PRs in 30d
Description
在分布式训练时，我准备了两个分片数据，启动两个worker，但其中一个worker读取数据报错：Graph build failed 。而使用这个节点进行一个worker训练时，则完全可以。请问这是什么原因造成的呢
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing distributed training with two data shards and two workers, then inspect the available logs around the “Graph build failed” message. Compare that run with single-worker training on the same node and identify the cause; done means the failure is explained and a verified fix or workaround is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100