necessary steps for syncing out-of-band upload of dandiset 108
- Dominant language
- Python
- Stars
- 28
- Forks
- 37
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 9
Description
for dandiset 108, we have been doing out of band uploads and recording locally the connection between asset id and zarr id. this was done because the asset ids disappeared for a batch of uploads at one point in the staging server.
at present:
- dandiset 108 contains both old assets and new assets. these are organized by sessions within folders.
- however, these new assets do not have appropriate metadata
- there are some assets that have not been uploaded. i would like to use the CLI to test upload performance of some of these.
questions:
1. what specific CLI steps need to be followed to sync the assets and dandisets?
2. can asset complete be forced through a debug flag if directly not available? or can the CLI force if it sees metadata does not match file count/size or is missing asset checksum?
3. how should i quantify operation times? are there debug flags that can be enabled to record time to check? (where operations include: are the files all available remotely, upload time, )
4. i am assuming that CLI upload is still slower than out of band upload + CLI check. has this been tested. if not can someone test this?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.