dandi / dandi/dandi-cli

organize: "dry run" on an external dandiset

Open
#114 0 comments 0 reactions 0 assignees View on GitHub
cmd-organize enhancement
Dominant language
Python
Stars
28
Forks
37
Avg merge
1d 17h
Merged PRs (30d)
9

Description

`organize` has (well, still has although unused) a `simulate` mode, and ability to load metadata from a .json (json lines) file:

```
if len(paths) == 1 and paths[0].endswith(".json"):
# Our dumps of metadata
metadata = load_jsonl(paths[0])
```

`download` already uses a helper to traverse an asset (dandiset or a folder) on dandiarchive: https://github.com/dandi/dandi-cli/blob/9bdb769430d6c9219b17db24438ecdbd51244608/dandi/girder.py#L200 and I believe full metadata records are included.

Thus it should be possible to provide functionality to
- get listing for all files/assets in dandiset
- run organize on that metadata
- compare new paths with original and report which would change.

That could be ran relatively quickly across **all** draft dandisets to see what would be effected whenever changes to `organize` logic would be introduced.

Shortcoming: metadata online might be lacking additional fields which we start to extract (e.g. `probe_ids`), so it might not provide 100% assurance, but at least it would be better than nothing ;)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.