karthik / karthik/smb_git

datasets as git repos: just files or also cataloging information/metadata?

Open
#22 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
86
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Great paper!

For the case of storing datasets in git, have you thought much about cataloging information or metadata about each dataset? I'm talking about metadata standards such as [Dublin Core](http://dublincore.org/documents/dcmi-terms/), [DataCite](http://schema.datacite.org/), [DDI](http://www.ddialliance.org/Specification/), et al. that would describe the dataset's title, subject, description, publisher, contributor, date, etc. in a standardized and interoperable manner (XML, JSON, etc.).

I know you're coming from the perspective of a researcher who is putting whatever you want in your local git repo, but imagine you'd like to eventually `git push` that dataset into a system that's a little more academically- and scientifically oriented than GitHub. What would your METADATA.json file look like? Would you expect the system receiving the `git push` to parse it and show all the metadata in its web interface?

All this is to say, should the metadata be part of the dataset git repo? What would a git repo for a dataset look like? I wrote up a thought experiment in the form of [a Google Doc](https://docs.google.com/document/d/18WDIS8hrFJvMJBcnRuQ8NfD-VxGq32vJ9WwlEgyyWZs/edit?usp=sharing) for anyone interested.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the issue text, its linked Google Doc, and the referenced Dublin Core, DataCite, and DDI standards. Determine whether dataset metadata belongs in the repository and what format or cataloging behavior is expected; done requires a documented, agreed direction rather than an implementation guess.

Written by the indexing model from the issue text.

Assessment

Tech stack
git
Domain
data, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.