JuliaML / JuliaML/MLDatasets.jl

Distribute datasets as artifacts

Open
#59 0 comments 1 reaction 0 assignees View on GitHub
help wanted
Dominant language
Julia
Stars
239
Forks
51
PR merge metrics
No merged PRs in 30d

Description

CRef: https://github.com/JuliaML/MLDatasets.jl/issues/57#issuecomment-809887071

We currently use DataDeps as an interface to download datasets from original websites. While it's good to give a clear license and source, it can be unstable for reproducibility because worldwide users might have difficulties connecting original sites. The original sites might also be offline for various reasons, e.g., #57.

To avoid issues like #57 in the future and accelerate dataset downloading, we could take advantage of Julia's [Artifacts](https://julialang.org/blog/2019/11/artifacts/) system and let Pkg/Storage servers hold and distribute the datasets. MLDatasets don't hold large datasets so it adds little stress to the Julia ecosystem.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.