cytomining / cytomining/profiling-template
Add instructions for using DVC for versioning data
- Dominant language
- No language data
- Stars
- 0
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
@gwaygenomics said this https://github.com/broadinstitute/lincs-cell-painting/pull/60#issuecomment-803462187
> We might at some point also consider moving from gitLFS to dvc. It was _super_ easy to get setup, and [plays very nicely with AWS](https://github.com/broadinstitute/grit-benchmark/pull/28/files#diff-53dfcd2d8adec5d69a09bdadd8051ea6252c8b99b9def70035aa4887364d322dR4). I did this in the grit-benchmark repo (in broadinstitute/grit-benchmark#28)
The [file pointer](https://github.com/gwaygenomics/grit-benchmark/blob/a04d010b2f579d5dd0cfdc2c9222c2d7f02b9a84/1.calculate-metrics/cell-health/results/cellhealth_single_cell_umap_embeddings_SQ00014610_chr2.tsv.gz.dvc) is in a readable format (YAML file)
```yaml
outs:
- md5: c53856c1596f00a67a636389716d8219
size: 26948901
path: cellhealth_single_cell_umap_embeddings_SQ00014610_chr2.tsv.gz
```
## Steps
1. Read the docs https://dvc.org/doc/start
1. Create a destination prefix (a "folder") on S3, which will be the remote storage location for dvc.
1. Add the `dvc` and `dvcs3` [dependencies](https://github.com/gwaygenomics/grit-benchmark/blob/a04d010b2f579d5dd0cfdc2c9222c2d7f02b9a84/environment.yml#L9-L10)
1. Update your `.gitignore` to [ignore the files](https://github.com/gwaygenomics/grit-benchmark/blob/a04d010b2f579d5dd0cfdc2c9222c2d7f02b9a84/1.calculate-metrics/cell-health/results/.gitignore) you used to previously track using GitLFS
1. Follow steps here https://dvc.org/doc/start and here https://dvc.org/doc/start/data-versioning
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.