MultiLayerFullNeighborSampler takes too much memory
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 🐛 Bug
Related discussion: https://discuss.dgl.ai/t/why-does-multilayerfullneighborsampler-consume-a-large-amount-of-memory/2454/12
MultilayerFullNeighborSampler used too much memory than expected. If change `MultilayerFullNeighborSampler(num_layers)` to `MultiLayerNeighborSampler([-1 for _ in range(num_layers)])`. The memory consumption goes much lower.
Potentially this might due to the implementation using in_subgraph but not sample_neighbors
Confirmed on the master branch
## To reproduce
using examples/pytorch/graphsage/train_sampling.py
Change sampler to `sampler = dgl.dataloading.MultiLayerFullNeighborSampler(4)` and `sampler = dgl.dataloading.MultiLayerNeighborSampler([-1 for _ in range(num_layers)])`.
cc @BarclayII
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.