Azure / Azure/azure-functions-dotnet-worker
ServiceBusTrigger with IsBatched true is delivering same message id more than once
- Dominant language
- C#
- Stars
- 466
- Forks
- 215
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 7
Description
### Description
.Net Framework: .net 8.0
Microsoft.Azure.Functions.Worker.Extensions.ServiceBus 5.22.0
We are writing an Azure function that will process many messages in batches but, for a single message in the queue, we are receiving multiple ServiceBusReceivedMessage with the same message id, and body but different delivery count (1,2,3). That is causing some issues as we would like to process batches of fixed-size to optimize costs on a dependency called by this function.
In addition to that, when we try to complete some of the messages, it will sometimes give us an Azure.Messaging.ServiceBus.ServiceBusException: "The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue". We think both problems are related since each duplicate message has a different lock token. We partially mitigated this by deduping messages by message id and doing a RenewMessageLockAsync(message) on every message before completing.
### Function signature looks as below with IsBatched flag as true:
`[Function("TestFunction")]
public async Task RunQueueAsync(
[ServiceBusTrigger("ourtopic", "oursubscription", Connection = "ServiceBusConfiguration:ConnectionString", **IsBatched = true, AutoCompleteMessages = false**)]
ServiceBusReceivedMessage[] messages,
ServiceBusMessageActions messageActions)`
### Host.json:
`{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
},
"enableLiveMetricsFilters": true
}
},
"functionTimeout": "00:30:00",
"extensions": {
"serviceBus": {
"autoCompleteMessages": false,
"autoRenewTimeout": "00:05:00",
"maxBatchWaitTime": "00:02:00",
"maxMessageBatchSize": 800,
"minMessageBatchSize": 80,
"maxAutoLockRenewalDuration": "00:30:00"
}
}
}`
### Logs:
`2024-09-17T21:30:13.837 [Information] Executing 'Functions.TestFunction' (Reason='(null)', Id=0d0777fe-d8f8-4ef9-a377-88f671929505)
2024-09-17T21:30:13.838 [Information] Trigger Details: MessageIdArray: 4ab8a173bf4a414b9bebd81ba5816eb3,4ab8a173bf4a414b9bebd81ba5816eb3,4ab8a173bf4a414b9bebd81ba5816eb3, SequenceNumberArray: 6,6,6, DeliveryCountArray: 1,2,3, EnqueuedTimeUtcArray: 2024-09-17T21:28:13.7660000+00:00,2024-09-17T21:28:13.7660000+00:00,2024-09-17T21:28:13.7660000+00:00, LockedUntilArray: 2024-09-17T21:29:13.7970000+00:00,2024-09-17T21:30:13.7700000+00:00,2024-09-17T21:31:13.7110000+00:00, SessionId: (null)`
### Steps to reproduce
1. Create function with IsBatched as true
2. Create a queue or topic (tried both)
3. Enqueue a single message
4. Wait for the time defined in maxBatchWaitTime
5. Check messages received in function
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the issue with a ServiceBusTrigger using IsBatched=true, AutoCompleteMessages=false, and the host.json batching settings shown. Compare the MessageIdArray, SequenceNumberArray, DeliveryCountArray, and lock data when one message is enqueued; done means a single queued message is not delivered repeatedly in one batch and completing received messages does not produce the reported lock exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100