Add versioning to all DGLDatasets to force reloading when codes are changed
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
Add versioning to all DGLDatasets to detect:
* Changes to the raw dataset files stored in S3. If so, re-download the dataset.
* Changes to the preprocessing code. If so, ignore the previous local cache and reprocess the data.
## Motivation
Brought up by #3987 which asks to revert the default reordering behavior of DGL builtin datasets. The issue is that even if we've implemented the request, users may still load cached datasets from local disk which may not reflect the latest change. Therefore, we require some versioning mechanism to detect those changes.
cc @mufeili
## Alternatives
Use a different dataset folder whenever DGL updates. This could cause excessive disk storage use.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.