dandi / dandi/dandi-cli

Validation error related to content-md5 when uploading OME-Zarr

Open
#1,821 19 comments 0 reactions 1 assignee Claimed by @yarikoptic View on GitHub
zarr
Dominant language
Python
Stars
28
Forks
37
Avg merge
1d 17h
Merged PRs (30d)
9

Description

Reposting an error from https://github.com/dandi/dandi-hub/issues/289#issuecomment-4082856320:

When running `dandi upload` on the DANDI Hub for an OME-Zarr asset the error below occurs.

@CodyCBakerPhD @jjnesbitt @yarikoptic Any suggestions? Thank you.

Tail of log:
```
2026-03-18T08:49:36-0500 [ERROR ] dandi 26563:140641760982784 Failed to upload /home/dstansby/data/linc/001411/sourcedata/raw/sub-I74/micr/sub-I74_sample-hemi_acq-zoom_chunk-02_XPCT.ome.zarr/0/c/0/0/186: Error 400 while sending PUT request to https://dandiarchive.s3.amazonaws.com/zarr/427e2ade-7a38-4f31-934b-6a74188120d6/0/c/0/0/186?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAUBRWC5GAEKH3223E%2F20260318%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20260318T131131Z&X-Amz-Expires=21600&X-Amz-SignedHeaders=content-md5%3Bhost%3Bx-amz-tagging&X-Amz-Signature=c8932c80b50642976753587f682fd6c9b2fb14a487506964d9099a7692e03bc3:
BadDigestThe Content-MD5 you specified did not match what we received.DvNGyXv56F0PdLculLqPdg==5a1b1a6ce22af597da3e7549b9100235SF2PBVY91D8WYS3W68PXcoPIpKamMD+KnrekF+CvXLZgCPccko/BW3bgsgJVjMWG/OHnc9pHx6HHr35or1aghLPWU16JcROUcGSCnM2F4vQhdSBU
2026-03-18T08:49:36-0500 [ERROR ] dandi 26563:140641760982784 Error uploading /home/dstansby/data/linc/001411/sourcedata/raw/sub-I74/micr/sub-I74_sample-hemi_acq-zoom_chunk-02_XPCT.ome.zarr:
Traceback (most recent call last):
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/upload.py", line 386, in process_path
for r in dfile.iter_upload(
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/files/zarr.py", line 813, in iter_upload
_handle_failed_items_and_raise(
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/files/zarr.py", line 903, in _handle_failed_items_and_raise
raise failed_items[0][1]
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/files/zarr.py", line 942, in _upload_zarr_file
storage_session.put(
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/dandiapi.py", line 325, in put
return self.request("PUT", path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/dandiapi.py", line 289, in request
raise requests.HTTPError(msg, response=result)
requests.exceptions.HTTPError: Error 400 while sending PUT request to https://dandiarchive.s3.amazonaws.com/zarr/427e2ade-7a38-4f31-934b-6a74188120d6/0/c/0/0/169?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAUBRWC5GAEKH3223E%2F20260318%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20260318T131131Z&X-Amz-Expires=21600&X-Amz-SignedHeaders=content-md5%3Bhost%3Bx-amz-tagging&X-Amz-Signature=6e3c6f4edb2aa4fa0d2f6a9c8b0426235a836aa8e2dc330a910a50506f325830:
BadDigestThe Content-MD5 you specified did not match what we received.ffRj2mH4Wvm8V8SHTc+k0w==d29ef077bb80cb57f588c15074f6582a5RWSTAAZ5WAH5E053FuSqNnmWJEa7oUosQztGcVUoVh4309SHbdy2Wt9z+zY73sRiOG+2v5OtnybkCbMxdHzhl+nb+fIliN1kuzBxHNLe1LiSqOU
2026-03-18T08:49:36-0500 [WARNING ] dandi 26563:140642653751104 One or more assets failed validation. Consult the logfile for details.
2026-03-18T08:49:36-0500 [DEBUG ] urllib3.connectionpool 26563:140642653751104 Starting new HTTPS connection (1): rig.mit.edu:443
2026-03-18T08:49:37-0500 [DEBUG ] dandi 26563:140642653751104 Caught exception failed validation
Traceback (most recent call last):
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/cli/base.py", line 126, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/cli/cmd_upload.py", line 107, in upload
upload(
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/upload.py", line 488, in upload
raise upload_err
File "/home/dstansby/envs/dandi/.venv/lib/python3.11/site-packages/dandi/upload.py", line 309, in process_path
raise UploadError("failed validation")
dandi.exceptions.UploadError: failed validation
2026-03-18T08:49:37-0500 [INFO ] dandi 26563:140642653751104 Logs saved in /home/dstansby/.local/state/dandi-cli/log/2026.03.18-12.34.26Z-26563.log
```

Full log: [2026.03.18-12.34.26Z-26563.log](https://github.com/user-attachments/files/26089224/2026.03.18-12.34.26Z-26563.log)

cc @satra @ayendiki @dstansby

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.