Azure / Azure/azure-sdk-for-python

[Cosmos] delete_all_items_by_partition_key returns 401 when container id contains a space (URL-encode mismatch in auth signing)

Open
#47,503 2 comments 0 reactions 0 assignees View on GitHub
bug Cosmos
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

### Summary

`container.delete_all_items_by_partition_key(...)` returns **401 Unauthorized** ("the expected payload is not built as per the protocol") when the container's `id` contains a URL-unsafe character (e.g. a space). The root cause is an **encoding mismatch between path construction and auth signing** in `_base.py`: the URL is URL-encoded but the resource link used to compute the auth signature is not.

Reproduces on a real production Cosmos account (`https://*.documents.azure.com`), not just PPE/staging.

### Repro

```python
import os, uuid
from azure.cosmos import CosmosClient, PartitionKey

client = CosmosClient(os.environ["COSMOS_ENDPOINT"], os.environ["COSMOS_KEY"])
db = client.create_database("repro_" + uuid.uuid4().hex[:8])

# container id contains a space
cid = "spaced id " + uuid.uuid4().hex[:8]
c = db.create_container(id=cid, partition_key=PartitionKey(path="/pk"))
c.upsert_item({"id": "i1", "pk": "p1"})

c.delete_all_items_by_partition_key("p1") # -> 401 Unauthorized
```

Output:

```
azure.cosmos.exceptions.CosmosHttpResponseError: (Unauthorized) The input authorization
token can't serve the request. The wrong key is being used or the expected payload is
not built as per the protocol.
Server used the following payload to sign:
'post
colls
dbs//colls/spaced%20id%20/operations/partitionkeydelete

'
```

Note the `%20` in the resource link the **server** used. The SDK signed with a raw space.

### Root cause

In `azure/cosmos/_cosmos_client_connection.py` (sync) and `azure/cosmos/aio/_cosmos_client_connection_async.py` (async) `DeleteAllItemsByPartitionKey`:

```python
path = base.GetPathFromLink(collection_link) # URL-encoded (%20)
path = '{}{}/{}'.format(path, "operations", "partitionkeydelete")
collection_id = base.GetResourceIdOrFullNameFromLink(collection_link) # NOT URL-encoded (raw space)
headers = base.GetHeaders(self, ..., "post", path, collection_id, "partitionkey", ...)
```

- `GetPathFromLink` (`_base.py:608`) calls `urllib_quote` → URL has `%20`.
- `GetResourceIdOrFullNameFromLink` (`_base.py:553`) only trims slashes → raw space.
- `auth.__get_authorization_token_using_master_key` signs the raw form.

The server reconstructs the canonical signing string from the request URL (encoded) and the two strings disagree → `(Unauthorized)`.

### Why it surfaced now

The pipeline test [`test_delete_all_items_by_partition_key`](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cosmos/azure-cosmos/tests/test_crud_response_payload_on_write_disabled.py) uses an id with a space:

```python
id='test_delete_all_items_by_partition_key ' + str(uuid.uuid4())
```

Until **PR #46568** (commit `7e47f0a194`, May 22, 2026) the test had this guard:

```python
# enable the test only for the emulator
if "localhost" not in self.host and "127.0.0.1" not in self.host:
return
```

That PR removed the guard. The emulator was tolerant of the encoding mismatch; real Cosmos isn't. First failing run: pipeline build `222650271` (sync + async test variants both fail, identical signature).

### Scope

Verified on a real account that the following ops on the same spaced-id container **do work**: `create_container`, `container.read()`, `upsert_item`, `read_item`, `database.delete_container`. The gateway evidently normalizes URL encoding for standard resource URLs but not for sub-operation paths like `…/operations/partitionkeydelete`. So today the user-visible impact is `delete_all_items_by_partition_key` only, but the SDK-side encoding mismatch is general and could bite any future sub-operation that follows the same pattern.

### Suggested fix

Either:

1. **(Preferred, smallest blast radius)** Make `GetResourceIdOrFullNameFromLink` URL-encode name-based links to match the URL path the server sees:

```python
if IsNameBased(resource_link):
return urllib_quote(TrimBeginningAndEndingSlashes(resource_link), safe="/")
```

This keeps SDK signing aligned with the URL the gateway receives.

2. Validate ids at the public API surface and reject names containing URL-encoding-significant characters. Backwards-incompatible.

### Regression test

Add (sync + async) coverage that creates a container with an id containing a space and runs `delete_all_items_by_partition_key`, alongside read/upsert/read-item to lock in the encoding contract across all container-scoped sub-operations.

### Repro environment

- Account: `https://*.documents.azure.com` (public production endpoint, master-key auth)
- SDK: `Azure/azure-sdk-for-python` `main` @ commit ~`ab7e36298b` (also reproduces on local checkout)
- Pipeline that surfaced it: `CosmosDB-SDK-SQL-Python-Signoff` build `222650271`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.