Azure / Azure/durabletask

Microsoft.Azure.ServiceBus.MessageSizeExceededException when a SubOrchestration returns data with size between a specific byte range

Open
#472 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
1.7k
Forks
335
Avg merge
2d 23h
Merged PRs (30d)
6

Description

In this scenario, we have an Orchestration calling a SubOrchestration that calls activities to compile the data that needs to be returned to the main orchestration.
**MainOrchestration->SubOrchestration->SameActivity many times**

At the end, the SubOrchestration returns an object containing an array of string (actually, GUIDs serialized to string).
For some reason, the durable task framework throws a **MessageSizeExceededException** when the array of the SubOrchestration contains between 3800 and 4200 members. Surprisingly, if the array contains less than 3800 items or more than 4200 members, the result is serialized and returned correctly to the main orchestration.

Since the issue was happening in a service, I wrote a small console app to reproduce it. Here is the code:
Microsoft.Azure.DurableTask.Core 2.4.1
Microsoft.Azure.DurableTask.ServiceBus 2.4.0
Microsoft.Azure.DurableTask.AzureStorage 1.8.1

MainOrchestration:

```
public class AnalyzeDataOrchestration : TaskOrchestration
{
public override async Task RunTask(OrchestrationContext context, AnalyzeDataParameters input)
{
CompileDataParameters parameters = new CompileDataParameters() { MaxResultsPerIteration = 1000, MaxIterations = 4 };
CompiledData data = await context.CreateSubOrchestrationInstance("CompileData", "1", parameters);

//TODO: Analyze data

return null;
}
}

```

SubOrchestration:

```
public class CompileDataSubOrchestration : TaskOrchestration
{
public override async Task RunTask(OrchestrationContext context, CompileDataParameters input)
{
List allData = new List();
for (int i = 0; i < input.MaxIterations; i++)
{
QueryDetails queryDetails = new QueryDetails() { IterationId = i, MaxResults = input.MaxResultsPerIteration };

QueryResult iterationResult = await context.ScheduleTask(typeof(QueryDataActivity), queryDetails);
allData.AddRange(iterationResult.Data);
}
return new CompiledData() { ObjectIds = allData.ToArray(), Iterations = input.MaxIterations };
}
}
public class CompileDataParameters
{
public int MaxIterations { get; set; }

public int MaxResultsPerIteration { get; set; }
}
public class CompiledData
{
public string[] ObjectIds { get; set; }

public int Iterations { get; set; }
}
```

Activity:
```
public class QueryDataActivity : TaskActivity
{
protected override QueryResult Execute(TaskContext context, QueryDetails input)
{
QueryResult result = new QueryResult { Data = new string[input.MaxResults] };
for (int i = 0; i < input.MaxResults; i++)
{
result.Data[i] = Guid.NewGuid().ToString();
}
return result;
}
}
public class QueryDetails
{
public int IterationId { get; set; }

public int MaxResults { get; set; }
}

public class QueryResult
{
public string[] Data { get; set; }
}
```

Here is the exception thrown by the SubOrchestration when the framework tries to return the result containing 4000 items in the array. (NB: 10 000 items works fine):

Exception: Microsoft.Azure.ServiceBus.MessageSizeExceededException : The received message (delivery-id:0, size:373117 bytes) exceeds the limit (262144 bytes) currently allowed on the link.
at Microsoft.Azure.ServiceBus.Core.MessageSender.d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.ServiceBus.RetryPolicy.d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.ServiceBus.RetryPolicy.d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.ServiceBus.Core.MessageSender.d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.ServiceBus.Common.Abstraction.MessageSender.d__6.MoveNext() in C:\source\durabletask\src\DurableTask.ServiceBus\Common\Abstraction\ServiceBusAbstraction.cs:line 612
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.ServiceBus.ServiceBusOrchestrationService.d__58.MoveNext() in C:\source\durabletask\src\DurableTask.ServiceBus\ServiceBusOrchestrationService.cs:line 821
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.Core.TaskOrchestrationDispatcher.d__21.MoveNext() in C:\source\durabletask\src\DurableTask.Core\TaskOrchestrationDispatcher.cs:line 460
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.Core.TaskOrchestrationDispatcher.d__20.MoveNext() in C:\source\durabletask\src\DurableTask.Core\TaskOrchestrationDispatcher.cs:line 123
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at DurableTask.Core.WorkItemDispatcher`1.d__45.MoveNext() in C:\source\durabletask\src\DurableTask.Core\WorkItemDispatcher.cs:line 375

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.