Azure Universal Artifacts download of large package fails
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
**Describe the bug**
Downloading a Universal Package of > 22GB fails when using az cli and self hosted agent.
**To Reproduce**
I uploaded a large universal package of around 22GB to a cloud hosted package feed. This took an appropriate amount of time based on my ISP upload speed. Using the following command then fails:
`az artifacts universal download --organization https://dev.azure.com/XXX --project="Test" --scope project --feed TestFeed --name ansible-media3 --version 2.3.2 --path fred`
It does seem to spend a good 30 minutes downloading what must be the majority of the package before failing with a 403 authorisation error.
**Expected behavior**
I would expect a universal package of any size up to the reported max of 1TB [(here)](https://docs.microsoft.com/en-us/azure/devops/artifacts/reference/limits?view=azure-devops&viewFallbackFrom=vsts) to download given sufficient time and space on the host.
**Environment summary**
Using Windows 10 powershell and:
azure-cli 2.18.0
core 2.18.0
telemetry 1.0.6
Extensions:
azure-devops 0.18.0
**Additional context**
There is a lot of information in an azure devops ticket [here](https://developercommunity2.visualstudio.com/t/azure-artifacts-download-of-large-package-fails/1308520?from=email&viewtype=all#T-ND1340546). A quick summary of what was done. We switched from using `az login` to using a PAT which didn't help. We then moved to self hosted agents which also gave the same error. We tried different accounts within the same organisation and this also didn't help.
In the same environment repeated downloads of 1GB and 4GB universal packages are fine.
My goal is to actually use universal packages of up to 100GB but this 22GB package was my third test of the capability.
We can't adopt Azure Universal Packages if the download fails.
I've attached [downloadexception.txt](https://github.com/Azure/azure-cli/files/5994348/downloadexception.txt) which was the initial error, [downloadwithpat.txt](https://github.com/Azure/azure-cli/files/5994352/downloadwithpat.txt) using a PAT and then [downloadlogwithdebug.txt](https://github.com/Azure/azure-cli/files/5994353/downloadlogwithdebug.txt) which has the debug flag on.
Have people downloaded large universal packages (>22GB) with the CLI?
I'm wondering if there is maybe some timeout option somewhere which means I get 30ish minutes of download before it fails?
Any help would be much appreciated. If this is the wrong place to raise this issue please let me know where to log it as the Azure Devops support after some time suggested here.
Contributor guide
Assessment
This issue has not been assessed yet.