Add documentation on how to produce DataLad BIDS dataset: 2 possible setups
- Dominant language
- Python
- Stars
- 5
- Forks
- 6
- Avg merge
- 4h 6m
- Merged PRs (30d)
- 1
Description
### approach 1: sourcedata/
Following the YODA style of containment and actual BIDS layout recommendations we could produce a BIDS dataset with
- sourcedata/
- `dandiset/` -- git submodule pointing to the dandiset from https://github.com/dandisets
then as a result of applying `nwb2bids` we could have multiple `--files-mode` (see dandi-cli which has/implemented them):
- `symlink` : would have `.nwb` files to become symlinks to original location, and if from under `sourcedata/` -- pointing to under `sourcedata/dandiset/...`
- (later) add a mode for `annex-copy` which would just copy the annex-key and establish the same annex key as original `sourcedata/dandiset/` and copy availability information (as we already have that file on s3)
Then if you use `datalad-fuse` on top I would expect it all work since datalad-fuse would provide "real file" view for the `sourcedata/dandiset/` dataset.
### approach 2: 3-branch method
A somewhat "slicker" alternative workflow could be similar to the one I demonstrated generated by datalad for "good old" openfmri datasets by our elderly https://github.com/datalad/datalad-crawler/blob/master/datalad_crawler/pipelines/simple_with_archives.py pipeline which produces 3 branches:
- `incoming` -- that is where it is the "original" content (here it would be current `draft` branch)
- `incoming-processed` -- that is where content of `incoming` for a new commit is processed into establishing a merge (it is not really a "merge" but rather: clean current worktree, take that tree of `incoming` and process (nwb2bids), and commit the result. (could be `draft-nwb2bids`)
- `master` -- true merges of the `incoming-processed` into `master` which could have its own content (e.g. `phenotype/` or `stimuli/` or 'code/') or changes (e.g. extended `dataset_description.json`) . (here could be just `bids` branch)
This way then it becomes feasible to inspect diffs in any of the branches or between them easily while git-annex would take care about efficient manipulation of content. It is quite amazing to be able to `git checkout bids` or `git checkout -` back to `draft` in a second or two.
edit: here is how it looks on https://datasets.datalad.org/openfmri/ds000001/.git repo in gitk:
Contributor guide
Assessment
This issue has not been assessed yet.