[GraphBolt] Dataset dtypes
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 🔨Work Item
**IMPORTANT:**
* This template is only for dev team to track project progress. For feature request or bug report, please use the corresponding issue templates.
* DO NOT create a new work item if the purpose is to fix an existing issue or feature request. We will directly use the issue in the project tracker.
Project tracker: https://github.com/orgs/dmlc/projects/2
## Description
Basically, any tensor that has indices corresponding to node ids should be stored with the int32 dtype.
- [ ] ogbn-arxiv dataset has the int64 dtypes while it can use int32 for csc_indptr and indices tensors and all the rest of the node ids tensors.
- [x] ogbn-papers100M has less than 2B nodes. Thus, we should store its indices array with the int32 dtype. Also, the train_set, validation_set and test_set should be stored with the int32 dtype as well.
- [x] mag240M dataset needs `indices`, `node_type_offset`, `train_set`, `validation_set`, `test_set` casted into int32. `type_per_edge` into uint8 or int8.
We don't need to do anything for `all_nodes_set` because it automatically gets its dtype from graph.indices.
### Saving features
When saving any feature tensors, we should make sure to use `gb.numpy_save_aligned` instead of `numpy.save`.
- [x] ogbn-papers100M (Thanks to @Liu-rj)
- [ ] ogb-lsc-mag240M
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.