pyout might be dragging us behind on large dandisets
- Dominant language
- Python
- Stars
- 28
- Forks
- 37
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 9
Description
While troubleshooting with @Kevancic the download of 000026 at MGH we observed that process is at 100% while not downloading much
py-spy top gave following

invocation was
```
DANDI_DEVEL_AGGRESSIVE_RETRY=1 dandi --log-level=INFO download -e refresh -J 1:1 https://dandiarchive.org/dandiset/000026/draft/files?location=sub-I48
```
note: we still were logging at DEBUG level in our log file, there were no pyout (?) logs there, but still were spending time validating using jsonschema every bloody record...
In general it should not be a problem but I think it is that within long running process and some "summary" or alike events start to pile up.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.