Azure / Azure/azure-storage-cpplite

Slow parallel upload using blob_client::upload_block_blob_from_buffer

Open
#96 1 comment 0 reactions 1 assignee Claimed by @katmsft View on GitHub
Dominant language
C++
Stars
26
Forks
50
PR merge metrics
No merged PRs in 30d

Description

When trying to upload especially smaller files using blob_client::upload_block_blob_from_buffer I find that upload speed is very slow (~500kbit/s). It appears that only a single parallel request is used although the maximum number of concurrent requests was set to 64.

From a cursory glance at the code, a rather large block size is chosen. That is, the minimum block size is set to azure::storage_lite::constants::default_block_size = 8 * 1024 * 1024, which is 8MB. Also, the grain size is set to around 4MB, where it is only 64kB in blob_client::download_blob_to_buffer.

Would it be feasible to set a smaller block size to allow for more blocks being uploaded in parallel even for smaller files?

Thanks for looking into this.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.