alibaba / alibaba/euler

Euler2.0分布式训练遇到的问题

Open
#295 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.9k
Forks
553
PR merge metrics
No merged PRs in 30d

Description

我使用cora的数据,在单机上跑通了graphsage模型。
使用中间生成的json文件(convert_data.json),执行了下面的命令
`sh gen_partitioned_data.sh ./cora/convert_data.json ./cora/meta ./temp/ 2 0`
请问

1、 这里的meta文件是在起什么作用?因为我看在cora的处理中,meta是空的。

2、在只有一个json文件的时候,执行这一个命令是否就可以把数据分成了2两个part

3、在接下来启动分布式训练的时候,zk_addr,zk_path,module,应该怎么填呢?分别起了什么样的作用。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading gen_partitioned_data.sh and the Cora convert_data.json workflow, then trace how the meta argument and partition count are consumed. Document the roles and expected values of meta, zk_addr, zk_path, and module, including how to verify that distributed training starts successfully.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.