microsoft / microsoft/AzureStorageExplorer
Default Transfer settings seem buggy/poorly chosen?
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 452
- Forks
- 92
- Avg merge
- 15h 20m
- Merged PRs (30d)
- 3
Description
Not a bug report yet - wanted to discuss it first. Happy to raise a proper bug report if it seems relevant after a brief discussion.
If uploading GBs of data in many files, via a low-upload-bandwidth connection the default Transfer settings seem not to work? Particularly the Network and File Concurrency.
I've got mass uploads working just fine, but only because I actively set those values both to be 1.
If I leave them blank (as is their default state) AzCopy attempts to upload 10s of 8MB chunks simultaneously, which ends up meaning that 90% of the chunks timeout and the copy spends its whole handling timeouts and re-sending the same chunks over and over.
Chunks do very occasionally get through, but it's pretty rare.
Conversely, when I limit the concurrency so that it's only uploading 1 chunk from 1 file at a time, it's saturating the network connection (assuming I leave Max Transfer Rate blank) and neatly uploading every file in order.
The defaults end up with a thing that doesn't work, and setting Concurrency to "1" works perfectly and is saturating my connection. So it's not like "it works but it's less efficient". We can't up load the data any quicker than the limit of the uplink connection!
Do you agree that (if my experience is representative) this is a bug and the defaults should be changed?
If so, I'll verify that I can reproduce, adn raise an actual bug.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names AzCopy transfer settings, especially Network and File Concurrency, but provides no source files, tests, or entry points. Start by reproducing a multi-file upload with the default settings versus both concurrency values set to 1; done means confirming the behavior and identifying whether the defaults should change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100