awslabs / awslabs/amazon-sqs-python-extended-client-lib
send_message_batch() fails when each message is below DEFAULT_MESSAGE_SIZE_THRESHOLD but in total are greater than 262144
- Dominant language
- Python
- Stars
- 52
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
### Description
When attempting to send a batch of messages where each message is below the DEFAULT_MESSAGE_SIZE_THRESHOLD but the total size exceeds 262144 bytes, boto3 sqs client's `send_message_batch()` function fails with a BatchRequestTooLong error.
### Steps to Reproduce
1. Initialize the SQS client with large payload support:
```python
import boto3
import sqs_extended_client
sqs_client: SQSClient = boto3.client('sqs')
sqs_client.large_payload_support = 'my_bucket_name'
queue_url = 'my_queue_url'
```
2. Attempt to send a batch of messages, each below the size threshold, but whose total size is above the size threshold:
```python
sqs_client.send_message_batch(QueueUrl=queue_url, Entries=[{'Id': '1','MessageBody':'A'*200000}, {'Id': '2','MessageBody':'B'*200000}])
```
### Expected Behavior
The batch of messages should be sent successfully.
### Actual Behavior
The operation fails with the following error:
```{BatchRequestTooLong}BatchRequestTooLong('An error occurred (AWS.SimpleQueueService.BatchRequestTooLong) when calling the SendMessageBatch operation: Batch requests cannot be longer than 262144 bytes. You have sent 400000 bytes.')```
### Additional Information
Just to verify all is configured properly on my setup, sending the large messages does work as intended, via the s3 bucket:
```python
sqs_client.send_message_batch(QueueUrl=queue_url, Entries=[{'Id': '1','MessageBody':'A'*400000}])
```
Contributor guide
Assessment
This issue has not been assessed yet.