datasets as git repos: just files or also cataloging information/metadata?
- Dominant language
- Shell
- Stars
- 86
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
Great paper!
For the case of storing datasets in git, have you thought much about cataloging information or metadata about each dataset? I'm talking about metadata standards such as [Dublin Core](http://dublincore.org/documents/dcmi-terms/), [DataCite](http://schema.datacite.org/), [DDI](http://www.ddialliance.org/Specification/), et al. that would describe the dataset's title, subject, description, publisher, contributor, date, etc. in a standardized and interoperable manner (XML, JSON, etc.).
I know you're coming from the perspective of a researcher who is putting whatever you want in your local git repo, but imagine you'd like to eventually `git push` that dataset into a system that's a little more academically- and scientifically oriented than GitHub. What would your METADATA.json file look like? Would you expect the system receiving the `git push` to parse it and show all the metadata in its web interface?
All this is to say, should the metadata be part of the dataset git repo? What would a git repo for a dataset look like? I wrote up a thought experiment in the form of [a Google Doc](https://docs.google.com/document/d/18WDIS8hrFJvMJBcnRuQ8NfD-VxGq32vJ9WwlEgyyWZs/edit?usp=sharing) for anyone interested.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the issue text, its linked Google Doc, and the referenced Dublin Core, DataCite, and DDI standards. Determine whether dataset metadata belongs in the repository and what format or cataloging behavior is expected; done requires a documented, agreed direction rather than an implementation guess.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git
- Domain
- data, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100