save changes to dandiset.yaml last
- Dominant language
- Python
- Stars
- 15
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
ATM, in case of crashes, like whenever some sha256 is not known for an asset, we seems to end up in a dirty dataset state
```
dandi@drogon:/mnt/backup/dandi/dandisets$ git status
On branch draft
Your branch is up to date with 'github/draft'.
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git checkout -- ..." to discard changes in working directory)
(commit or discard the untracked or modified content in submodules)
modified: 000026 (modified content)
no changes added to commit (use "git add" and/or "git commit -a")
dandi@drogon:/mnt/backup/dandi/dandisets$ cd 000026/
dandi@drogon:/mnt/backup/dandi/dandisets/000026$ git status
On branch draft
Your branch is up to date with 'github/draft'.
Changes to be committed:
(use "git reset HEAD ..." to unstage)
modified: dandiset.yaml
```
since it seems changes to dandiset.yaml are saved first. Would it be easy to save them last, right before calling datalad's save on the dandiset? Then there would be a better chance to remain in a clean state for subsequent cron job to possibly succeed.
Contributor guide
No contributing guide indexed for this repository
Research direction
Locate the workflow in this repository that writes each dandiset.yaml and then calls DataLad's save, starting with the code responsible for processing asset sha256 values. Trace the ordering around a crash caused by an unknown sha256. Done means dandiset.yaml is saved only immediately before the dandiset save, reducing the chance that a failed run leaves the dataset dirty.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100