Odd crash on some file with spaces in the name in 001449
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 5
Description
```
dandi@drogon:/mnt/backup/dandi/dandisets$ duct --f-t 0 bash -c '/mnt/backup/dandi/dandisets/tools/backups2datalad-update-cron 001449'
2026-05-28T09:08:04-0400 [INFO ] con-duct: python-dotenv not installed, skipping .env file loading
2026-05-28T09:08:04-0400 [INFO ] con-duct: duct 0.20.1 is executing 'bash -c /mnt/backup/dandi/dandisets/tools/backups2datalad-update-cron 001449'...
2026-05-28T09:08:04-0400 [INFO ] con-duct: Log files will be written to .duct/logs/2026.05.28T09.08.04-1450355_
2026-05-28T09:08:30-0400 [ERROR ] backups2datalad: Job failed on input :
+ Exception Group Traceback (most recent call last):
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/aioutil.py", line 219, in dowork
| outp = await func(inp)
| ^^^^^^^^^^^^^^^
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/datasetter.py", line 214, in update_dandiset
| changed = await self.sync_dataset(dandiset, ds, dmanager)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/datasetter.py", line 294, in sync_dataset
| await syncer.sync_assets()
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/syncer.py", line 128, in sync_assets
| report = await async_assets(
| ^^^^^^^^^^^^^^^^^^^
| self.dandiset, self.ds, self.manager, self.tracker, self.error_on_change
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| )
| ^
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/asyncer.py", line 509, in async_assets
| anyio.create_task_group() as nursery,
| ~~~~~~~~~~~~~~~~~~~~~~~^^
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 815, in __aexit__
| raise BaseExceptionGroup(
| "unhandled errors in a TaskGroup", self._exceptions
| )
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/asyncer.py", line 307, in process_blob
| raise RuntimeError(
| ...<2 lines>...
| )
| RuntimeError: derivatives/atlas_registration/sub-A1/ses-20240227T172507/micr/sub-A1_ses-20240227T172507_sample-brain_stain-cFos_space-Unified mouse brain atlas (Kim lab)_desc-atlas_cells.tsv identified as text but is 13812195 bytes!
+------------------------------------
2026-05-28T09:08:30-0400 [ERROR ] backups2datalad: An error occurred:
Traceback (most recent call last):
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/__main__.py", line 126, in wrapped
await f(datasetter, *args, **kwargs)
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/__main__.py", line 256, in update_from_backup
await datasetter.update_from_backup(dandisets, exclude=exclude)
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/datasetter.py", line 157, in update_from_backup
raise RuntimeError(
f"Backups for {quantify(len(report.failed), 'Dandiset')} failed"
)
RuntimeError: Backups for 1 Dandiset failed
Logs saved to /mnt/backup/dandi/dandisets/.git/dandi/backups2datalad/2026/05/2026.05.28.13.08.13Z.log
2026-05-28T09:08:30-0400 [INFO ] con-duct: Summary:
Exit Code: 1
Command: bash -c /mnt/backup/dandi/dandisets/tools/backups2datalad-update-cron 001449
Log files location: .duct/logs/2026.05.28T09.08.04-1450355_
Wall Clock Time: 26.469 sec
Memory Peak Usage (RSS): 141.4 MB
Memory Average Usage (RSS): 110.8 MB
Virtual Memory Peak Usage (VSZ): 532.7 MB
Virtual Memory Average Usage (VSZ): 331.5 MB
Memory Peak Percentage: 0.10%
Memory Average Percentage: 0.07%
CPU Peak Usage: 132.40%
Average CPU Usage: 53.08%
```
likely simply due to the space in the filename!:
```
❯ curl --silent -X 'GET' 'https://api.dandiarchive.org/api/dandisets/001449/versions/draft/assets/?path=derivatives%2Fatlas_registration%2Fsub-A1%2Fses-20240227T172507%2Fmicr%2F&metadata=false&zarr=false' -H 'accept: application/json' | jq -r '.results[].path' | grep sub-A1_ses-20240227T172507_sample-brain_stain-cFos_
derivatives/atlas_registration/sub-A1/ses-20240227T172507/micr/sub-A1_ses-20240227T172507_sample-brain_stain-cFos_desc-annotCountsLeaf.csv
derivatives/atlas_registration/sub-A1/ses-20240227T172507/micr/sub-A1_ses-20240227T172507_sample-brain_stain-cFos_space-Unified mouse brain atlas (Kim lab)_desc-atlas_cells.json
derivatives/atlas_registration/sub-A1/ses-20240227T172507/micr/sub-A1_ses-20240227T172507_sample-brain_stain-cFos_space-Unified mouse brain atlas (Kim lab)_desc-heatmap_SPIM.json
derivatives/atlas_registration/sub-A1/ses-20240227T172507/micr/sub-A1_ses-20240227T172507_sample-brain_stain-cFos_space-Unified mouse brain atlas (Kim lab)_desc-atlas_cells.tsv
derivatives/atlas_registration/sub-A1/ses-20240227T172507/micr/sub-A1_ses-20240227T172507_sample-brain_stain-cFos_space-Unified mouse brain atlas (Kim lab)_desc-heatmap_SPIM.tif
```
so we need to check that all git/git-annex invocations are done via list of args and if string invocation is needed, we quote correctly (using a `shlex.quote()` or alike)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.