alibaba / alibaba/euler

分布式训练ps负载不均衡

Open
#254 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.9k
Forks
553
PR merge metrics
No merged PRs in 30d

Description

我用自己写的模型,在分布式下跑,10个分片,9千万结点,启用2个ps会oom,但是启用3个ps时第三个启用了但是没有数据进入。
测试了用line去跑一份简单的数据,3个ps1个worker,只有第一个ps工作,第23个ps没有参与计算,该如何解决?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported setup with 10 shards, 90 million nodes, and two versus three parameter servers, then compare it with the simpler Line dataset using three parameter servers and one worker. Done means identifying why only the first parameter server receives work and documenting a validated fix or clear reproduction findings.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.