awslabs / awslabs/amazon-sqs-python-extended-client-lib

send_message_batch() fails when each message is below DEFAULT_MESSAGE_SIZE_THRESHOLD but in total are greater than 262144

Open
#25 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
52
Forks
7
PR merge metrics
No merged PRs in 30d

Description

### Description
When attempting to send a batch of messages where each message is below the DEFAULT_MESSAGE_SIZE_THRESHOLD but the total size exceeds 262144 bytes, boto3 sqs client's `send_message_batch()` function fails with a BatchRequestTooLong error.

### Steps to Reproduce
1. Initialize the SQS client with large payload support:
```python
import boto3
import sqs_extended_client
sqs_client: SQSClient = boto3.client('sqs')
sqs_client.large_payload_support = 'my_bucket_name'
queue_url = 'my_queue_url'
```
2. Attempt to send a batch of messages, each below the size threshold, but whose total size is above the size threshold:
```python
sqs_client.send_message_batch(QueueUrl=queue_url, Entries=[{'Id': '1','MessageBody':'A'*200000}, {'Id': '2','MessageBody':'B'*200000}])
```
### Expected Behavior
The batch of messages should be sent successfully.

### Actual Behavior
The operation fails with the following error:
```{BatchRequestTooLong}BatchRequestTooLong('An error occurred (AWS.SimpleQueueService.BatchRequestTooLong) when calling the SendMessageBatch operation: Batch requests cannot be longer than 262144 bytes. You have sent 400000 bytes.')```

### Additional Information
Just to verify all is configured properly on my setup, sending the large messages does work as intended, via the s3 bucket:
```python
sqs_client.send_message_batch(QueueUrl=queue_url, Entries=[{'Id': '1','MessageBody':'A'*400000}])
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.