dandi / dandi/dandi-archive

Dandiset "modified" timestamp not updated when entries deleted from Zarr

Open
#1,871 0 comments 0 reactions 0 assignees View on GitHub
bug zarr
Dominant language
Python
Stars
26
Forks
21
Avg merge
4d 23h
Merged PRs (30d)
15

Description

When one or more entries are deleted from a Zarr via the `DELETE /zarr/{zarr_id}/files/` endpoint — even if the Zarr is then reingested by calling `POST /zarr/{zarr_id}/finalize/` — the "modified" properties of both the containing Dandiset and Dandiset version are not updated.

MVCE:

```python
import os
from pathlib import Path
from tempfile import TemporaryDirectory
from time import sleep
from dandi.consts import dandiset_metadata_file
from dandi.dandiapi import DandiAPIClient
from dandi.files import dandi_file
import numpy as np
import zarr

with DandiAPIClient.for_dandi_instance(
"dandi-staging", token=os.environ["DANDI_API_KEY"]
) as client:

d = client.create_dandiset(
"Test Zarr Dandiset",
{
"schemaKey": "Dandiset",
"name": "Test Dandiset",
"description": "A test Dandiset",
"contributor": [
{
"schemaKey": "Person",
"name": "Wodder, John",
"roleName": ["dcite:Author", "dcite:ContactPerson"],
}
],
"license": ["spdx:CC0-1.0"],
},
)

dandiset_id = d.identifier
print("DANDISET ID:", dandiset_id)
try:
with TemporaryDirectory() as tmpdir:
dspath = Path(tmpdir)
(dspath / dandiset_metadata_file).write_text(
f"identifier: '{dandiset_id}'\n"
)
zarr.save(dspath / "sample.zarr", np.arange(1000), np.arange(1000, 0, -1))
df = dandi_file(dspath / "sample.zarr", dandiset_path=dspath)
print("Uploading Zarr ...")
df.upload(d, {"path": "sample.zarr", "description": "A Zarr"})

d.refresh()
print("Dandiset modified at:", d.modified)
print("Dandiset draft version modified at:", d.version.modified)

print("Sleeping ...")
sleep(3)

(asset,) = d.get_assets()
entry = next(asset.iterfiles())
print("Deleting entry", entry, "from Zarr")
asset.rmfiles([entry])

d.refresh()
print("Dandiset modified at:", d.modified)
print("Dandiset draft version modified at:", d.version.modified)
finally:
print("Deleting Dandiset ...")
d.delete()
```

Sample output:

```text
DANDISET ID: 212987
Uploading Zarr ...
Dandiset modified at: 2024-02-26 14:06:12.328706+00:00
Dandiset draft version modified at: 2024-02-26 14:06:12.527791+00:00
Sleeping ...
Deleting entry .zgroup from Zarr
Dandiset modified at: 2024-02-26 14:06:12.328706+00:00
Dandiset draft version modified at: 2024-02-26 14:06:12.527791+00:00
Deleting Dandiset ...
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the DELETE /zarr/{zarr_id}/files/ endpoint and POST /zarr/{zarr_id}/finalize/ entry points, using the supplied Python MVCE against dandi-staging. Confirm that deleting Zarr entries updates the containing Dandiset and Dandiset version modified timestamps, including after finalization.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.