A few questions for the usage
- Dominant language
- Python
- Stars
- 72
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
It's really fantastic! Thank you so much for sharing this project.
I had a quick test with `minio` docker process and confirmed it works really well as expected.
I'd like to ask a few questions about the usage:
- can I save any objects other than tensors, i.e. a tuple of tensors, a dict or a sparse tensors? In this case how to give the attributes?
- If I add some more data samples to an existing dataset (in the case that periodically data samples could be added so if I have to refresh all the datasets with the added samples), will it be okay to add to the dataset and save it?
- If I use this in a distributed training, will the each dataloader occupy the amount of dataset? For example, if I'd like to use this with PyTorch DDP, then each process will have its own `DataLoader` and will load the data samples during the training -- so I wonder the loaded dataset size will be multiple of the original dataset size per process or not.
- It looks like supporting only PyTorch and TensorFlow now, but how about the numpy array or matrix for scikit-learn or XGBoost? Can I store some numpy objects as well?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.