dandi / dandi/dandisets

error_on_change can be "racy"

Open
#246 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
15
Forks
2
PR merge metrics
No merged PRs in 30d

Description

NB I hit `Enter` my accident before filling it in

This is a follow up to #245 PR which introduces default behavior of quick checks of dandisets by their modification times to decide either to skip or go through assets and reflect changes. In the mode of `--verify-timestamps` it might happen that we would be going through assets **while** dandiset gets updated, and by the time we get to some asset the done previously check based solely on date would no longer be valid. So I think that may be `error_on_change` should actually be a callback/fixture for a specific dandiset to re-ask that dandiset modification datetime, so if we detected some change, and see that modification time did change from previous one -- we are all good an do not need to error out on that run for that or any subsequent check like that.

Since situation is still hypothetical, I decided to just file an issue instead of requesting changes to that PR. But if you @jwodder think it is worthwhile implementing right there -- go ahead.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading PR #245 and tracing the implementation of --verify-timestamps and error_on_change. Determine how a dandiset modification-time change during asset checks should affect the run, then document and test the agreed behavior so the race case has a clear outcome.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.