dmlc / dmlc/dgl

[RFC] Customizability enhancement of graph sampling pipeline

Open
#3,600 0 comments 1 reaction 0 assignees View on GitHub
RFC
Dominant language
Python
Stars
14.3k
Forks
3.1k
PR merge metrics
No merged PRs in 30d

Description

Author: @jermainewang @BarclayII @nv-dlasalle
Status: Draft | **Discussion** | Approved | Dropped | Suspended
Full doc: https://docs.google.com/document/d/105TMN9Jb5yiHpfG8bESmRCHU1KgfW_QzTx5vYcWVi_A/

Motivation
---
* Support sampling from graphs stored in external non-DGL sources (e.g., cuGraph, graph DB, etc.)
* Support fetching features from external non-DGL storage (e.g., CUDA Unified Virtual Address (UVA) tensor, multi-GPU tensor, KVStore, TorchQuiver, etc.)
* Users only need to focus on the sampling logic while most of the system optimizations (e.g., lazy feature copy, async data movement) can be largely reused.

Brief Summary of the Technical Approach
---
The idea is to have an agreement on the interface for graph structure storage as well as feature storage. In the future, DGL will refactor its sampling pipeline according to the proposed interface so that users with custom data storage can easily leverage DGL's sampling algorithms without any internal codebase change.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.