Azure / Azure/azure-sdk-for-python

azure-eventhub consumer stops processing events after `Received message 257:None is not WSMsgType.BINARY`

Open
#43,073 3 comments 0 reactions 1 assignee Claimed by @j7nw4r View on GitHub
Client customer-reported Event Hubs needs-team-attention question Service Attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

- **Package Name**: `azure-eventhub`
- **Package Version**: `5.15.0`
- **Operating System**: Debian, from [python:3.12-slim docker image](https://github.com/docker-library/python/blob/093598a0190ba9074b899d6a0a21a00c859aac56/3.12/slim-trixie/Dockerfile)
- **Python Version**: `3.12`

**Describe the bug**
An Async Eventhub consumer **completely stopped processing events** after it hit this exception:

```
"EventProcessor instance '27933907-a7f5-4a14-8405-4d298e948c46' of eventhub 'bankupdates_v2' partition '0' consumer group '$default'. An error occurred while receiving. The exception is ConnectError(\"Can not read frame due to exception: Websocket disconnected: WSMessageTypeError('Received message 257:None is not WSMsgType.BINARY')\\nError condition: ErrorCondition.SocketError\\n Error Description: Can not read frame due to exception: Websocket disconnected: WSMessageTypeError('Received message 257:None is not WSMsgType.BINARY')\")."
```

We saw one more log message after this from our `on_partition_error` handler:

```"
An exception: 0 occurred during receiving from Partition: Can not read frame due to exception: Websocket disconnected: WSMessageTypeError('Received message 257:None is not WSMsgType.BINARY')\nError condition: ErrorCondition.SocketError\n Error Description: Can not read frame due to exception: Websocket disconnected: WSMessageTypeError('Received message 257:None is not WSMsgType.BINARY').
```

And we saw this one from our `on_partition_close` handler:

```
""
```

**To Reproduce**
Steps to reproduce the behavior:
1. good question!

**Expected behavior**
(no bugs forever ;) )

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**
We found this related (seemingly similar?) issue for `azure-servicebus`, fixed earlier this year: https://github.com/Azure/azure-sdk-for-python/issues/36280

For further context (some of which may be red herrings), I can offer the following:
- This application uses Avro serialization for its events.
- The Eventhub is a Standard SKU (with 1mb event size max)
- The events themselves are **large** (we don't see this in our other apps with very small events). The **publisher** of these events sometimes sees rejections for sending events that are too large. I don't know if this is involved in this issue, but figured it was worth sharing.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.