Dataloading requires 5x the memory of the indices tensor when a dictionary is passed
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 🐛 Bug
When running UVA training on an edge-centric dataset, and doing sampling on a subset of indces, these lines require 5x the size of the indices tensor worth of GPU memory, causing OOMs with medium sized datasets:
https://github.com/dmlc/dgl/blob/master/python/dgl/dataloading/dataloader.py#L88-L95
## To Reproduce
Steps to reproduce the behavior:
1. Create a moderately large graph (in my case ~ 1B edges)
2. Create formats for the graph (required for UVA)
3. Create a sampler that samples most of those edges (with UVA on), with indices stored on GPU.
4. Run sampling
## Expected behavior
Ideally, if we need to create these tensors, they should be stored in host memory. The best case scenario is that we can somehow do this operation in place.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.