Azure / Azure/Azurite

delete_blobs method raises an exception PartialBatchErrorException when using Azurite

Open
#1,809 6 comments 3 reactions 1 assignee Claimed by @EmmaZhu View on GitHub
blob-storage bug
Dominant language
TypeScript
Stars
2.3k
Forks
393
Avg merge
1d 20h
Merged PRs (30d)
36

Description

### Which service(blob, file, queue, table) does this issue concern?
blob

### Which version of the Azurite was used?
3.22.0

### Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
DockerHub

### What's the Node.js version?
NA
### What problem was encountered?
Azurite returns an error when calling `delete_blobs()`
```
Traceback (most recent call last):
File "/home/myLocalPath/.config/JetBrains/PyCharm2022.3/scratches/azurite.py", line 30, in
container_client.delete_blobs(*blob_list)
File "/home/myLocalPath/.virtualenvs/py310/lib/python3.10/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/myLocalPath/.virtualenvs/py310/lib/python3.10/site-packages/azure/storage/blob/_container_client.py", line 1360, in delete_blobs
return self._batch_send(*reqs, **options)
File "/home/myLocalPath/.virtualenvs/py310/lib/python3.10/site-packages/azure/storage/blob/_shared/base_client.py", line 319, in _batch_send
process_storage_error(error)
File "/home/myLocalPath/.virtualenvs/py310/lib/python3.10/site-packages/azure/storage/blob/_shared/response_handlers.py", line 181, in process_storage_error
exec("raise error from None") # pylint: disable=exec-used # nosec
File "", line 1, in
File "/home/myLocalPath/.virtualenvs/py310/lib/python3.10/site-packages/azure/storage/blob/_shared/base_client.py", line 315, in _batch_send
raise error
azure.storage.blob._shared.response_handlers.PartialBatchErrorException: There is a partial failure in the batch operation.
ErrorCode:None
Content: --batch_7240c4fb-b11c-11ed-ac16-d39098213827
Content-Type: application/http

HTTP/1.1 400 One of the request inputs is not valid.
x-ms-error-code: InvalidInput
x-ms-request-id: 89d6790c-18d2-4a2c-88b3-80c73e5614ab
content-type: application/xml

InvalidInput
One of the request inputs is not valid.
RequestId:89d6790c-18d2-4a2c-88b3-80c73e5614ab
Time:2023-02-20T12:45:25.460Z

--batch_7240c4fb-b11c-11ed-ac16-d39098213827--
```
### Logs
Debug logs: [debug.log](https://github.com/Azure/Azurite/files/10794259/debug.log)
```
Azurite Blob service is starting on 0.0.0.0:10000
Azurite Blob service successfully listens on http://0.0.0.0:10000
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container?restype=container HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "GET /devstoreaccount1/container?restype=container&comp=list HTTP/1.1" 200 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob0 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob1 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob2 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob3 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob4 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob5 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob6 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob7 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob8 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "PUT /devstoreaccount1/container/my_blob9 HTTP/1.1" 201 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "GET /devstoreaccount1/container?restype=container&comp=list HTTP/1.1" 200 -
172.17.0.1 - - [20/Feb/2023:12:45:25 +0000] "POST /devstoreaccount1/container?restype=container&comp=batch HTTP/1.1" 202 -
```

### Steps to reproduce the issue?
I used script below to test it with azurite and my storage account. Azurite sends 400 in response, but on storage account the script works fine and deletes blobs.
```
from azure.core.exceptions import ResourceExistsError

from azure.storage.blob import BlobServiceClient

blob_service_client = BlobServiceClient.from_connection_string((
"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;"
"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/"
"K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"
))

container_client = blob_service_client.get_container_client("container")

try:
container_client.create_container()
except ResourceExistsError:
pass

blob_list = [b.name for b in list(container_client.list_blobs())]
# empty list
print(blob_list)

upload_data = b"Hello World"
for index in range(10):
container_client.upload_blob(name=f"my_blob{index}", data=upload_data, overwrite=True)

# uploaded blobs
blob_list = [b.name for b in list(container_client.list_blobs())]
print(blob_list)

container_client.delete_blobs(*blob_list)

# empty again
blob_list = [b.name for b in list(container_client.list_blobs())]
print(blob_list)
```

### Have you found a mitigation/solution?
In patch side effect I iterate and remove single blob. It's global patch in TestRunner
```
def delete_azure_blobs(self, *blobs, **kwargs):
for blob in blobs:
with suppress(ResourceNotFoundError):
self.delete_blob(blob)

@patch(
"azure.storage.blob._container_client.ContainerClient.delete_blobs",
side_effect=delete_azure_blobs,
autospec=True,
)
def run_tests(self, test_labels, mock, **kwargs):
return super().run_tests(test_labels, **kwargs)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.