[RFC] Customizability enhancement of graph sampling pipeline
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
Author: @jermainewang @BarclayII @nv-dlasalle
Status: Draft | **Discussion** | Approved | Dropped | Suspended
Full doc: https://docs.google.com/document/d/105TMN9Jb5yiHpfG8bESmRCHU1KgfW_QzTx5vYcWVi_A/
Motivation
---
* Support sampling from graphs stored in external non-DGL sources (e.g., cuGraph, graph DB, etc.)
* Support fetching features from external non-DGL storage (e.g., CUDA Unified Virtual Address (UVA) tensor, multi-GPU tensor, KVStore, TorchQuiver, etc.)
* Users only need to focus on the sampling logic while most of the system optimizations (e.g., lazy feature copy, async data movement) can be largely reused.
Brief Summary of the Technical Approach
---
The idea is to have an agreement on the interface for graph structure storage as well as feature storage. In the future, DGL will refactor its sampling pipeline according to the proposed interface so that users with custom data storage can easily leverage DGL's sampling algorithms without any internal codebase change.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.