dmlc / dmlc/dgl

[Bug] COOToCSR() fails with unexpected error message for negative node ids

Open
#3,613 1 comment 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
14.3k
Forks
3.1k
PR merge metrics
No merged PRs in 30d

Description

## 🐛 Bug

If you create a DGL graph with negative node ids in COO format (eg due to unexpected integer overflow), and attempt to convert it to CSR via this:
```
# Create DGL graph.
# Note: src/dst_node_ids contains negative values, eg due to integer overflow
# this is a Tensor with shape=[num_edges] where each edge is (src_node_ids_tensor[i], dst_node_ids_tensor[i])
src_node_ids_tensor = torch.Tensor(..., dtype=torch.int32)
dst_node_ids_tensor = torch.Tensor(..., dtype=torch.int32)
graph_data_dict: Dict[Tuple[str, str, str], Tuple[torch.Tensor, torch.Tensor]] = OrderedDict([
# (edge_relation, tuple_of_node_tensors)
(
("node_type_a", "edge_relation_00", "node_type_b"),
(src_node_ids_tensor, dst_node_ids_tensor),
),
])
dgl_graph = dgl.heterograph(graph_data_dict)
# Convert from COO to CSR, and strip out COO format to reduce memory
dgl_graph = dgl_graph.formats(['csr'])
```
you'll get the following error:

Stack trace:
```
Traceback (most recent call last):
File "my_script.py", line 114, in
main()
File "my_script.py", line 77, in main
dgl_graph = dgl_graph.formats(['csr'])
File "/usr/local/lib/python3.7/site-packages/dgl/heterograph.py", line 5708, in formats
ret._graph = self._graph.formats(formats)
File "/usr/local/lib/python3.7/site-packages/dgl/heterograph_index.py", line 971, in formats
return _CAPI_DGLHeteroGetFormatGraph(self, formats)
File "dgl/_ffi/_cython/./function.pxi", line 287, in dgl._ffi._cy3.core.FunctionBase.__call__
File "dgl/_ffi/_cython/./function.pxi", line 222, in dgl._ffi._cy3.core.FuncCall
File "dgl/_ffi/_cython/./function.pxi", line 211, in dgl._ffi._cy3.core.FuncCall3
File "dgl/_ffi/_cython/./base.pxi", line 155, in dgl._ffi._cy3.core.CALL
dgl._ffi.base.DGLError: [21:59:08] /dgl/src/array/cpu/spmat_op_impl_coo.cc:599: Check failed: Bp[N] == NNZ (-1868267179 vs. 2426700117) :
Stack trace:
[bt] (0) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x75) [0x7f0cc606ab95]
[bt] (1) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(+0x32fd33) [0x7f0cc620cd33]
[bt] (2) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(dgl::aten::CSRMatrix dgl::aten::impl::COOToCSR<(DLDeviceType)1, int>(dgl::aten::COOMatrix)+0x59) [0x7f0cc620d0b2]
[bt] (3) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(dgl::aten::COOToCSR(dgl::aten::COOMatrix)+0x180) [0x7f0cc604bb50]
[bt] (4) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(dgl::UnitGraph::GetOutCSR(bool) const+0x155) [0x7f0cc64a8e4f]
[bt] (5) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(dgl::UnitGraph::GetGraphInFormat(unsigned char) const+0x2f5) [0x7f0cc64aca17]
[bt] (6) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(dgl::HeteroGraph::GetGraphInFormat(unsigned char) const+0x1d1) [0x7f0cc6388bcd]
[bt] (7) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(+0x4c4f51) [0x7f0cc63a1f51]
[bt] (8) /usr/local/lib/python3.7/site-packages/dgl/libdgl.so(std::function::operator()(dgl::runtime::DGLArgs, dgl::runtime::DGLRetValue*) const+0x22) [0x7f0cc631e9a2]
```

## To Reproduce

I don't have a formal reproduction script yet, if you'd like me to produce one I can. I happened to run into this because I was trying to create a DGL graph with 4853400234 (~4.85B edges), but I had incorrectly casted my graph_data_dict Tensors to int32 (rather than int64), which led to int overflow resulting in negative node ids.

## Expected behavior

Ideally, DGL code can quickly+efficiently catch the negative-node-id issue earlier, probably at the initial graph creation time. Otherwise, the negative-node-id issue can cascade into difficult-to-debug error messages (like in my scenario).

## Environment

- DGL Version (e.g., 1.0): https://github.com/dmlc/dgl/commit/cd6d113874119c2bde7262c27421268b86f2e9de
- Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3):
- OS (e.g., Linux): Linux
- How you installed DGL (`conda`, `pip`, source): source
- Build command you used (if compiling from source):
- Python version: 3.7
- CUDA/cuDNN version (if applicable):
- GPU models and configuration (e.g. V100):
- Any other relevant information:

## Additional context

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.