dandi / dandi/dandi-archive

Zarr stuck in "Ingesting"

Open
#2,219 8 comments 0 reactions 0 assignees View on GitHub
zarr
Dominant language
Python
Stars
26
Forks
21
Avg merge
4d 23h
Merged PRs (30d)
15

Description

Reported by @gmazzamuto , one of the authors of the 000026, that he fails to update one asset , even fails to delete it
https://api.dandiarchive.org/api/dandisets/000026/versions/draft/assets/fc330324-6693-48cd-87aa-9192ab9caeaf/

looking at the asset, indeed that Zarr seems got stuck in Ingesting state:

```shell
❯ curl --silent -X 'GET' 'https://api.dandiarchive.org/api/zarr/23123eef-62df-46b8-92ae-c2102110977c/' | jq .
{
"name": "sub-I48/ses-SPIM/micr/sub-I48_ses-SPIM_sample-BrocaAreaS04_stain-NeuN_SPIM.ome.zarr",
"dandiset": "000026",
"zarr_id": "23123eef-62df-46b8-92ae-c2102110977c",
"status": "Ingesting",
"checksum": null,
"file_count": 0,
"size": 0
}
```

when I tried to send DELETE request too in the web interface, got `HTTP 204 No Content` (so it seems succeded)

![Image](https://github.com/user-attachments/assets/11c46cb5-e0d5-46a8-8026-2f3bf4592d3a)

and subsequent repeated DELETE - 404, so seems like I did manage to delete it

![Image](https://github.com/user-attachments/assets/5fe4abab-6982-4cc1-9c65-7b0e0edb8f54)

The https://api.dandiarchive.org/api/dandisets/000026/versions/draft/assets/fc330324-6693-48cd-87aa-9192ab9caeaf/ is still showing the asset but

atpath says not found

```shell
❯ curl -i -X 'GET' 'https://api.dandiarchive.org/api/webdav/assets/atpath/?dandiset_id=000026&version_id=draft&path=sub-I48%2Fses-SPIM%2Fmicr%2Fsub-I48_ses-SPIM_sample-BrocaAreaS04_stain-NeuN_SPIM.ome.zarr'
HTTP/1.1 404 Not Found
Report-To: {"group":"heroku-nel","max_age":3600,"endpoints":[{"url":"https://nel.heroku.com/reports?ts=1742307466&sid=c46efe9b-d3d2-4a0c-8c76-bfafa16c5add&s=pL562qIu%2FawpFJsf5aSTNFUunwj6yK1lpZVRliVnprw%3D"}]}
Reporting-Endpoints: heroku-nel=https://nel.heroku.com/reports?ts=1742307466&sid=c46efe9b-d3d2-4a0c-8c76-bfafa16c5add&s=pL562qIu%2FawpFJsf5aSTNFUunwj6yK1lpZVRliVnprw%3D
Nel: {"report_to":"heroku-nel","max_age":3600,"success_fraction":0.005,"failure_fraction":0.05,"response_headers":["Via"]}
Connection: keep-alive
Server: gunicorn
Date: Tue, 18 Mar 2025 14:17:46 GMT
Content-Type: application/json
Vary: Accept, Cookie, origin
Allow: OPTIONS, GET
X-Frame-Options: DENY
Content-Length: 125
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
Via: 1.1 vegur

{"detail":"Folder or asset not found at sub-I48/ses-SPIM/micr/sub-I48_ses-SPIM_sample-BrocaAreaS04_stain-NeuN_SPIM.ome.zarr"}
```

so we can assume I did annihilate it.

@gmazzamuto could you please reattempt upload of that zarr and report here how it fails if it does (may be share the dandi client log)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reattempting the Zarr upload for dandiset 000026 and capture the DANDI client log, as requested in the issue. Compare the Zarr status endpoint, asset endpoint, and WebDAV atpath response; done means the upload does not remain in Ingesting and asset deletion or visibility is consistent across these endpoints.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.