How to use dgl with hdfs?
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
hi, the User Guide says the function dgl.load_graphs() supports "loading the graphs directly from hdfs (by providing "hdfs://..." a path)", I try it and get the following results (here 10.242.187.67 is the ip address of namenode in my hdfs) :
The Trackback shows I need to compile with DMLC_USE_HDFS=1 to use hdfs, but when I compile DGL with dgl/script/build_dgl.sh, I don't find anywhere to add this flag.
Can someone tell me the correct way to use dgl with hdfs?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the User Guide section describing dgl.load_graphs() and dgl/script/build_dgl.sh, then compare the documented HDFS behavior with the traceback shown in the issue. Determine how DMLC_USE_HDFS=1 is intended to be enabled and update the relevant usage or build documentation so the documented HDFS path can be followed and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100