Azure / Azure/azure-storage-python

BlockBlobService.get_blob_to_path() crashes even when if_match='*'

Open
#625 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
343
Forks
243
PR merge metrics
No merged PRs in 30d

Description

### Which service(blob, file, queue) does this issue concern?
BlockBlobService.get_blob_to_path()

### Which version of the SDK was used? Please provide the output of `pip freeze`.
Same issue is encountered in versions: 0.37.1, 1.3.0, and 2.1.0

**Version 0.37.1**
>pip freeze
azure-common==1.1.8
azure-nspkg==2.0.0
azure-storage==0.36.0
azure-storage-blob==0.37.1
azure-storage-common==0.37.1
azure-storage-nspkg==2.0.0

**Version 1.3.0**
>pip freeze
azure-common==1.1.8
azure-nspkg==2.0.0
azure-storage==0.36.0
azure-storage-blob==1.3.0
azure-storage-common==1.3.0
azure-storage-nspkg==2.0.0

**Latest version**
>pip freeze
azure-common==1.1.8
azure-nspkg==2.0.0
azure-storage==0.36.0
azure-storage-blob==2.1.0
azure-storage-common==2.1.0
azure-storage-nspkg==2.0.0

### What problem was encountered?
When trying to download a blob that is currently been written in append mode (i.e., the ETag changes during time of download), `BlockBlobService.get_blob_to_path()` crashes even when using `if_match='*'` giving **ConditionNotMet** Error:
`Client-Request-ID=9cce10c8-b739-11e9-9366-b32a86ae676b Retry policy did not allow for a retry: Server-Timestamp=Mon, 05 Aug 2019 04:29:28 GMT, Server-Request-ID=86bb1780-a01e-002e-6146-4b01e3000000, HTTP status code=412, Exception=The condition specified using HTTP conditional header(s) is not met. ErrorCode: ConditionNotMetConditionNotMetThe condition specified using HTTP conditional header(s) is not met.RequestId:86bb1780-a01e-002e-6146-4b01e3000000Time:2019-08-05T04:29:28.3320963Z`

### Have you found a mitigation/solution?
- Using a version before **1.3.1** and using `max_connections=1` mitigates the problem
- Downloading a very small portion of the blob (< 20 MB) will work since ETag doesn't change during time of download

Code to repro:
```
bbs = BlockBlobService(connection_string=connection_string)
bbs.get_blob_to_path(app_id, blob_name, output_fp, start_range=0, end_range=100*1024**2, if_match='*')
```

Contributor guide

Open the contributing guide

Research direction

Start at BlockBlobService.get_blob_to_path() and run the supplied 100 MiB download reproduction with if_match='*' while the append blob is being written. Compare the behavior across the listed SDK versions; done means the download no longer crashes with ConditionNotMet under the stated conditions, with regression coverage for the reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.