dandi / dandi/dandisets

Propose users to provide their own data availability resources to be registered within DataLad dandisets

Open
#274 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
15
Forks
2
PR merge metrics
No merged PRs in 30d

Description

In principle could be done via ContentUrls at the level of dandischema and thus centralized at the archive level.
But first the idea came up in context of datalad, might be easier to "implement" via datalad which has hooks into various other data portals. It was inspired by questions about data longevity we received today from ncsu.edu .

We could allow people to report availability of data in form of
- dictionary/list of sha256 checksum: URL
- URL to any "git remote" compatible git/git-annex remote (e.g. on gin, or on any git-annex special remote via git datalad remote)

then we could easily register those within datalad dandisets we have. In principle some of that information could even be reflected in metadata records for assets (if direct urls) or dandisets (full remotes, so individual urls might be tricky/impossible).

- not quite "IPFS" but not intended to be ;)
- might be trickier/too cumbersome for collections of zarr dandisets but still doable

WDYT @satra ?

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files, tests, or entry points. Start by locating the DataLad integration and dandiset metadata registration paths, then clarify the supported checksum-to-URL and git-remote cases and how successful registration should be represented.

Written by the indexing model from the issue text.

Assessment

Tech stack
git
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.