Uploading the same Zarr assets to different embargoed Dandisets requires data transfer both times
- Dominant language
- Python
- Stars
- 28
- Forks
- 37
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 9
Description
Hi @yarikoptic, I uploaded a set of Zarr assets to [Dandiset 1602](https://dandiarchive.org/dandiset/001602) and am now uploading the same assets to [Dandiset 1411](https://dandiarchive.org/dandiset/001411). The terminal output seems to suggest that data is being transferred again during the upload to the second Dandiset. My understanding is that if the assets already exist on the Archive and they are being uploaded again, then a link would just be created to the existing assets from the second Dandiset and actual data transfer would not occur. Thanks.
Terminal output:
```bash
03:04 $ DANDI_DEVEL=1 dandi upload --allow-any-path --validation ignore .
2025-09-19 03:05:16,249 [ INFO] Found 15 files to consider
PATH SIZE ERRORS PROGRESS STATUS MESSAGE
dandiset.yaml 1.3 kB skipped should be edited online
...b-I74/micr/sub-I74_sample-IC_acq-overview_XPCT.json 77 Bytes 0 done
...4/micr/sub-I74_sample-IC_acq-overview_XPCT.ome.zarr 89.9 GB 0 42% uploading
.../micr/sub-I74_sample-IC_acq-zoom_chunk-01_XPCT.json 74 Bytes 0 done
...r/sub-I74_sample-IC_acq-zoom_chunk-01_XPCT.ome.zarr 277.4 GB 0 1% uploading
.../micr/sub-I74_sample-IC_acq-zoom_chunk-02_XPCT.json 74 Bytes 0 done
...r/sub-I74_sample-IC_acq-zoom_chunk-02_XPCT.ome.zarr 341.1 GB 0 0% uploading
.../micr/sub-I74_sample-IC_acq-zoom_chunk-03_XPCT.json 74 Bytes 0 done
...r/sub-I74_sample-IC_acq-zoom_chunk-03_XPCT.ome.zarr 134.7 GB 0 17% uploading
...I74/micr/sub-I74_sample-hemi_acq-overview_XPCT.json 77 Bytes 0 done
...micr/sub-I74_sample-hemi_acq-overview_XPCT.ome.zarr 274.5 GB 0 4% uploading
...icr/sub-I74_sample-hemi_acq-zoom_chunk-01_XPCT.json
...sub-I74_sample-hemi_acq-zoom_chunk-01_XPCT.ome.zarr
...icr/sub-I74_sample-hemi_acq-zoom_chunk-02_XPCT.json
...sub-I74_sample-hemi_acq-zoom_chunk-02_XPCT.ome.zarr
Summary: 1.1 TB 101.4 MB/s 1 skipped 1 should be edited online
5 done
5 uploading
```
cc @satra
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.