anthropics / anthropics/anthropic-sdk-python

Bedrock streaming SSE events errors are not handled gracefully

Open
#1,477 0 comments 0 reactions 0 assignees View on GitHub
bedrock
Dominant language
Python
Stars
3.9k
Forks
853
Avg merge
1d 18h
Merged PRs (30d)
11

Description

In some cases bedrock SSE stream produce error SSE events like following:
```
{'status_code': 400, 'headers': {':exception-type': 'internalServerException', ':content-type': 'application/json', ':message-type': 'exception'}, 'body': b'{"message":"The system encountered an unexpected error during processing. Try your request again."}
```

**Current behaviour**
Anthropic SDK raises `ValueError`
```python
File "/app/.venv/lib/python3.12/site-packages/anthropic/_streaming.py", line 211, in __aiter__
async for item in self._iterator:
File "/app/.venv/lib/python3.12/site-packages/anthropic/_streaming.py", line 225, in __stream__
async for sse in iterator:
File "/app/.venv/lib/python3.12/site-packages/anthropic/_streaming.py", line 215, in _iter_events
async for sse in self._decoder.aiter_bytes(self.response.aiter_bytes()):
File "/app/.venv/lib/python3.12/site-packages/anthropic/lib/bedrock/_stream_decoder.py", line 50, in aiter_bytes
message = self._parse_message_from_event(event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/anthropic/lib/bedrock/_stream_decoder.py", line 58, in _parse_message_from_event
raise ValueError(f"Bad response code, expected 200: {response_dict}")
ValueError: Bad response code, expected 200: {'status_code': 400, 'headers': {':exception-type': 'internalServerException', ':content-type': 'application/json', ':message-type': 'exception'}, 'body': b'{"message":"The system encountered an unexpected error during processing. Try your request again."}'}
```

**Expected behaviour**
SDK should either retry this internally or propagate to user-facing `APIStatusError`

**Environment**
anthropic SDK v0.96.0
python 3.12

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.