dandi / dandi/dandi-schema

Add asset "layout" metadata field, validate asset paths

Open
#157 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7
Forks
15
PR merge metrics
No merged PRs in 30d

Description

in a follow up to https://github.com/dandi/dandi-cli/issues/1205#issuecomment-1421010945 of @satra

> (@yarikoptic - this was done outside of dandi schema and also related to web-based validation, rather than local validation).

yes, that was done in https://github.com/dandi/dandi-cli/pull/1173 but it does not mean that it should stay here, at least as the "implementation logic".
In the past we removed `dandi-cli` from dependencies of `dandi-archive` IIRC to break the "dependency loop" since `dandi-cli` depends on API implementation of the archive. Hence we keep code worth reusing in both -cli and -archive elsewhere (dandi-schema, zarr_checksum) etc.

So may be we should move the [validate_organized_path](https://github.com/dandi/dandi-cli/blob/master/dandi/organize.py#L1028) into dandischema? But again -- then for an asset it would matter on either it is a BIDS asset or some other (DANDI layout) asset.

looking at a sample asset from 000026 which is BIDS -- we do not provide such metadata

```shell
❯ curl --silent -X 'GET' 'https://api.dandiarchive.org/api/assets/3a6cdce9-ab49-41dd-967f-29349d0cc341/' -H 'accept: application/json' | jq .
{
"id": "dandiasset:3a6cdce9-ab49-41dd-967f-29349d0cc341",
"path": "sub-EXC022/ses-MRI/anat/sub-EXC022_ses-MRI-echo-2_flip-2_VFA.json",
"access": [
{
"status": "dandi:OpenAccess",
"schemaKey": "AccessRequirements"
}
],
"digest": {
"dandi:sha2-256": "f48296e2812476e813c8d11e59ab1947ddfc6a8330c32ace04f040cd5d7d213e",
"dandi:dandi-etag": "a7aaf0935f14bdb4d686486f4512a299-1"
},
"@context": "https://raw.githubusercontent.com/dandi/schema/master/releases/0.6.0/context.json",
"schemaKey": "Asset",
"contentUrl": [
"https://api.dandiarchive.org/api/assets/3a6cdce9-ab49-41dd-967f-29349d0cc341/download/",
"https://dandiarchive.s3.amazonaws.com/blobs/5d5/2c6/5d52c625-2efd-4ff2-a768-bd4f15e7da46"
],
"identifier": "3a6cdce9-ab49-41dd-967f-29349d0cc341",
"repository": "https://dandiarchive.org/",
"contentSize": 247,
"dateModified": "2021-10-05T14:50:57.025495-04:00",
"schemaVersion": "0.6.0",
"encodingFormat": "application/json",
"wasGeneratedBy": [
{
"id": "urn:uuid:7eec0bb1-668c-48da-a3be-a90df5485d8c",
"name": "Metadata generation",
"schemaKey": "Activity",
"description": "Metadata generated by DANDI cli",
"wasAssociatedWith": [
{
"url": "https://github.com/dandi/dandi-cli",
"name": "DANDI Command Line Interface",
"version": "0.27.3",
"schemaKey": "Software",
"identifier": "RRID:SCR_019009"
}
]
}
],
"blobDateModified": "2021-10-05T12:55:21.243891-04:00"
}

```

so to validate the path we should first start annotating assets with "layout" (optional, default to DANDI, could be BIDS). Then validate either using our DANDI layout regex, or BIDS (using bidsschematools like done in dandi-cli) purely on the path (ref also #74 ).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.