Kaszanas / Kaszanas/SC2_Datasets
HuggingFace Dataset Interfaces
Open
feature request
good first issue
- Dominant language
- Python
- Stars
- 17
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
HuggingFace is a very nice way of hosting datasets.
What is even more interesting is that larger StarCraft 2 datasets can contain more than 3TB of data.
HugginfFace answers this with the ability to stream the data.
https://github.com/huggingface/datasets
https://huggingface.co/docs/datasets/stream
This needs to be looked into, it might be better to transform all of the JSON datasets into a format that leverages the HuggingFace exposed APIs for ease of access and compatibility.
Contributor guide
Assessment
This issue has not been assessed yet.