Azure / Azure/azure-sdk-for-python
[EventHub] sending large messages >1MB in dedicated clusters fails
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
When dedicated EH clusters are used, max message size can be set up to 20MB. Currently, when the client is negotiating with the service, it sets a default value of 1MB as the max message size on the outgoing ATTACH frame on the sender link. The service then accepts the 1MB as the max message size and returns the same value in the incoming ATTACH, even when the max message size in the cluster is configured to 20MB.
To ensure that the configured max message size is used, the client should be setting the max message size to 0 on the outgoing ATTACH on the sender link. The service will then return the configured max message size on the hub. This follows the .NET client behavior (which can be reproduced by running the [.NET sample with amqp proxy](https://github.com/richardpark-msft/amqpfaultinjector/blob/main/samples/faultinjectorsample_net/Program.cs).)
TODO:
- [ ] Update [this](https://github.com/Azure/azure-sdk-for-python/blob/ff3e1ff1dcee5d368267ab13fb2c995e44d5e025/sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/client.py#L579) in both sync and async SendClients AND ReceiveClients to set default value to 0 instead of MAX_FRAME_SIZE. i.e. `self._max_message_size = kwargs.pop("max_message_size", 0)`
- [ ] Make the same change in Service Bus vendored _pyamqp to ensure consistency + that nothing breaks.
- [ ] Test by sending individual messages > 1MB and batch messages > 1MB. Both work in .NET and should work in Python.
Contributor guide
Assessment
This issue has not been assessed yet.