[GraphBolt] GraphBolt objects cannot be serialized during multiprocessing
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 🐛 Bug
It seems that GraphBolt objects like `SampledSubgraph` cannot be passed around in multiprocessing due to lack of serialization method.
## To Reproduce
```python
import dgl.graphbolt
import torch
import multiprocessing as mp
import scipy.sparse as sp
def rand_csc_graph(N, density):
adj = sp.random(N, N, density)
adj = adj + adj.T
adj = adj.tocsc()
indptr = torch.LongTensor(adj.indptr)
indices = torch.LongTensor(adj.indices)
graph = dgl.graphbolt.from_csc(indptr, indices)
return graph
def _entry(graph, q):
adjs = []
seeds = torch.arange(5)
for hop in range(2):
sg = graph.sample_neighbors(seeds, torch.LongTensor([2]))
seeds = sg.indices
adjs.insert(0, sg)
print('Putting')
q.put(adjs)
print('Done put')
if __name__ == '__main__':
graph = rand_csc_graph(200, 0.15)
q = mp.Queue()
proc = mp.Process(target=_entry, args=(graph, q))
proc.start()
print('Getting')
item = q.get()
print('Done get')
proc.join()
print(item)
```
This code snippet hangs indefinitely, either silently or throws the following error:
```
Traceback (most recent call last):
File "/home/ubuntu/miniconda3/lib/python3.9/multiprocessing/queues.py", line 245, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/ubuntu/miniconda3/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
RuntimeError: Tried to serialize object __torch__.torch.classes.graphbolt.SampledSubgraph which does not have a __getstate__ method defined!
```
## Expected behavior
The data should be successfully printed. Replacing `q.put(adjs)` to something else like `q.put(42)` works. So it should be something explicit to GraphBolt.
## Environment
- DGL Version (e.g., 1.0): master
- Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3):
- OS (e.g., Linux):
- How you installed DGL (`conda`, `pip`, source):
- Build command you used (if compiling from source):
- Python version:
- CUDA/cuDNN version (if applicable):
- GPU models and configuration (e.g. V100):
- Any other relevant information:
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.