[RFC][DataLoader] Support `sample_blocks` API for non-DGLGraph objects
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
We should add support to `sample_blocks` API for non-DGLGraph objects to use in the sampling pipeline.
This will follow how we currently support `sample_neighbors` for Non-DGL Graph objects.
Document: https://hackmd.io/6PJixJj7Ssy19I-N9CV1Ag#Extending-DGL-Graph-Sampling-Pipeline
Related Issue: https://github.com/dmlc/dgl/issues/3600
## Motivation
Currently, in sample blocks (see below), we iterate for each value of `fanout.`
Though this approach works when the data is present locally when we want to do this through an external service (say something like `GAAS`) this can become a bottleneck as we will need to transfer graphs back and forth from the service for each value of fanout.
This will mean we do a bunch of calls like below which can become really slow.
```
External Sampling Process -> Training Process -> External Sampling Process -> .....
```
https://github.com/dmlc/dgl/blob/d41d07d0f6cbed17993644b58057e280a9e8f011/python/dgl/dataloading/neighbor_sampler.py#L106-L120
## Pitch
We will have to modify the current [sample_blocks](https://github.com/dmlc/dgl/blob/2b76674002cb1df4de237024838513b6679dfbf2/python/dgl/dataloading/neighbor_sampler.py#L106-L120) to check if the underlying object has an attribute for `sample_blocks` and if yes call that directly and if not call the above iterative sample_neighbors.py
## Additional context
Related issue: https://github.com/dmlc/dgl/issues/4326
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.