Conflict on lock while updating 000571
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 5
Description
```
dandi@drogon:/mnt/backup/dandi/dandisets/000571$ [locked backup2datalad]$ /mnt/backup/dandi/dandisets/tools/backups2datalad-update-cron 000571
fatal: Unable to create '/mnt/backup/dandi/dandisets/000571/.git/index.lock': File exists.
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
fatal: Unable to create '/mnt/backup/dandi/dandisets/000571/.git/index.lock': File exists.
...
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
fatal: Unable to create '/mnt/backup/dandi/dandisets/000571/.git/index.lock': File exists.
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
2025-06-16T18:31:59-0400 [ERROR ] backups2datalad: Job failed on input :
Traceback (most recent call last):
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/aioutil.py", line 177, in dowork
outp = await func(inp)
^^^^^^^^^^^^^^^
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/datasetter.py", line 192, in update_dandiset
changed = await self.sync_dataset(dandiset, ds, dmanager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/datasetter.py", line 244, in sync_dataset
await syncer.sync_assets()
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/syncer.py", line 77, in sync_assets
report = await async_assets(
^^^^^^^^^^^^^^^^^^^
self.dandiset, self.ds, self.manager, self.tracker, self.error_on_change
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/asyncer.py", line 612, in async_assets
await ds.commit(
...<2 lines>...
)
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/adataset.py", line 341, in commit
raise RuntimeError(
...<2 lines>...
)
RuntimeError: /mnt/backup/dandi/dandisets/000571 is still dirty after committing. Please check if all changes were staged.
2025-06-16T18:31:59-0400 [ERROR ] backups2datalad: An error occurred:
Traceback (most recent call last):
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/__main__.py", line 119, in wrapped
await f(datasetter, *args, **kwargs)
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/__main__.py", line 229, in update_from_backup
await datasetter.update_from_backup(dandisets, exclude=exclude)
File "/home/dandi/miniconda3/envs/dandisets-2/lib/python3.13/site-packages/backups2datalad/datasetter.py", line 135, in update_from_backup
raise RuntimeError(
f"Backups for {quantify(len(report.failed), 'Dandiset')} failed"
)
RuntimeError: Backups for 1 Dandiset failed
Logs saved to /mnt/backup/dandi/dandisets/.git/dandi/backups2datalad/2025/06/2025.06.16.21.59.04Z.log
^Cndi@drogon:/mnt/backup/dandi/dandisets/000571$ [locked backup2datalad]$ ^C
dandi@drogon:/mnt/backup/dandi/dandisets/000571$ [locked backup2datalad]$ ls -ld /mnt/backup/dandi/dandisets/000571/.git/index.lock
ls: cannot access '/mnt/backup/dandi/dandisets/000571/.git/index.lock': No such file or directory
```
possibly related
- #42 where it is on zarr
note: it is quite a heavy update with lots of `.mefd/` folders in this BIDS dandiset
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing backups2datalad-update-cron 000571 with the heavy .mefd/ workload, then read datasetter.py, syncer.py, asyncer.py, and adataset.py at the stack-trace locations. Done means the update handles the reported index-lock condition and does not leave the dandiset dirty after the operation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100