JuliaML / JuliaML/MLDatasets.jl
Distribute datasets as artifacts
- Dominant language
- Julia
- Stars
- 239
- Forks
- 51
- PR merge metrics
- No merged PRs in 30d
Description
CRef: https://github.com/JuliaML/MLDatasets.jl/issues/57#issuecomment-809887071
We currently use DataDeps as an interface to download datasets from original websites. While it's good to give a clear license and source, it can be unstable for reproducibility because worldwide users might have difficulties connecting original sites. The original sites might also be offline for various reasons, e.g., #57.
To avoid issues like #57 in the future and accelerate dataset downloading, we could take advantage of Julia's [Artifacts](https://julialang.org/blog/2019/11/artifacts/) system and let Pkg/Storage servers hold and distribute the datasets. MLDatasets don't hold large datasets so it adds little stress to the Julia ecosystem.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.