box / box/box-python-sdk

Retry-After header is ignored because response headers are stored in a case-sensitive dict

Open
#1,582 0 comments 0 reactions 5 assignees Claimed by @mwwoda View on GitHub
bug
Dominant language
Python
Stars
459
Forks
223
Avg merge
8h 57m
Merged PRs (30d)
13

Description

- [x] I have checked that the SDK documentation doesn't solve my issue.
- [x] I have checked that the API documentation doesn't solve my issue.
- [x] I have searched the Box Developer Forums and my issue isn't already reported.
- [x] I have searched Issues in this repo and my issue isn't already reported.

### Description of the Issue

`BoxRetryStrategy` looks up the `Retry-After` header with exact casing, but response headers are stored in a plain `dict` that preserves whatever casing the server sent. Box's API returns this header lowercased (`retry-after`), so the lookup never matches and the SDK silently falls back to its own exponential backoff, discarding the wait time the server asked for.

`BoxNetworkClient` converts requests' `CaseInsensitiveDict` into a plain `dict` (both occurrences):

```python
# box_sdk_gen/networking/box_network_client.py
headers=dict(response.network_response.headers),
```

`BoxRetryStrategy` then queries that dict with capitalized keys:

```python
# box_sdk_gen/networking/retries.py
retry_after_header: Optional[str] = (
fetch_response.headers.get('Retry-After')
if 'Retry-After' in fetch_response.headers
else None
)
```

Both the `in` check and the `.get()` are case-sensitive, so neither matches `retry-after`.

This affects two code paths in `retries.py`:

1. `retry_after()` -- the delay requested by the server is discarded and replaced with the SDK's own exponential backoff.
2. `should_retry()` -- `is_accepted_with_retry_after` never becomes `True`, so the `202 Accepted` + `Retry-After` retry path does not trigger either.

### Steps to Reproduce

The mismatch can be demonstrated without any Box credentials:

```python
from requests.structures import CaseInsensitiveDict

# What requests hands to the SDK (Box sends the header lowercased)
headers = CaseInsensitiveDict({'retry-after': '90'})
print(headers.get('Retry-After')) # '90' - fine while it is a CaseInsensitiveDict

# What the SDK actually stores and queries
plain = dict(headers) # box_network_client.py
print(list(plain.keys())) # ['retry-after']
print('Retry-After' in plain) # False <-- the guard in retries.py
print(plain.get('Retry-After')) # None <-- the lookup in retries.py
```

End to end: trigger any response that carries `retry-after` (for example a `503` while creating many folders in quick succession) and observe that the gap between retries follows `2**attempt * retry_base_interval * random(0.5, 1.5)` rather than the value the server sent.

### Expected Behavior

The `Retry-After` value is honored regardless of header casing, and a `202 Accepted` carrying `Retry-After` is retried as intended.

### Actual Behavior

`Retry-After` is never found. With the default `BoxRetryStrategy` (`max_attempts=5`, `retry_base_interval=1`), a server asking for a 90 second pause is instead retried after roughly 1-3s, 2-6s, 4-12s and 8-24s, exhausting every attempt in about 15-45 seconds. The client retries hardest at exactly the moment the service asked it to back off.

### Error Message, Including Stack Trace

Redacted response from a `POST /2.0/folders` that failed this way. Note the lowercase `retry-after` in the header dump the SDK itself prints:

```
Message: 503 Service is temporarily unavailable
Request:
Method: POST
URL: https://api.box.com/2.0/folders
Response:
Status code: 503
Headers:
{ 'cache-control': 'no-cache, no-store',
'content-type': 'application/json',
'retry-after': '90',
'x-envoy-upstream-service-time': '5181'}
Code: unavailable
```

### Suggested Fix

Either keep the headers case-insensitive when building `FetchResponse`:

```python
headers=CaseInsensitiveDict(response.network_response.headers),
```

or make the lookup itself case-insensitive, which avoids depending on the network client implementation:

```python
retry_after_header = next(
(v for k, v in fetch_response.headers.items() if k.lower() == 'retry-after'),
None,
)
```

As a side note, `float(retry_after_header)` raises `ValueError` if the header arrives in the HTTP-date form that RFC 9110 also permits.

### Versions Used

Python SDK: reproduced with `box-sdk-gen` 1.17.0, where the same lookup exists in a simpler form (`fetch_response.headers.get('Retry-After')` without the `in` guard). The code quoted above is from this repository's `main`, which has the same defect.
Python: 3.10.8

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.