Azure / Azure/durabletask

Timeouts finish unexpectedly when events are fired inside an orchestration

Open
#636 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
1.7k
Forks
335
Avg merge
2d 23h
Merged PRs (30d)
6

Description

As part of a demo, I have a sub-orchestration that simulates a 2 step communication with a 3rd party. (sends a signal and then waits for a callback until continuing with the execution. The issue is that every time an event is raised, the timeOut suddenly completes during the initial replay (so before the onEvent was processed) and the suborchestration fails due to this.
**I'm using the SQLProvider for DTF.**

Sample code :
Sub orchestration :
```

public class CompatibilityOrchestrator : TaskOrchestration
{
TaskCompletionSource receivedCompatResponseEvent = new TaskCompletionSource();
private readonly ILogger _logger;

public CompatibilityOrchestrator(ILogger logger)
{
_logger = logger;
}
public override async Task RunTask(OrchestrationContext context, CompatibilityGenerationRequest input)
{
var sv = input;
sv.OrchestrationInstanceID = context.OrchestrationInstance.InstanceId;
_logger.LogInformation("Sending generate compatibility report signal to 3rd Party :" + sv.OrchestrationInstanceID.ToString());

var timeoutTask = context.CreateTimer(context.CurrentUtcDateTime.AddMinutes(5), "TimedOut");

var winner = await Task.WhenAny(receivedCompatResponseEvent.Task, timeoutTask);

if (winner == receivedCompatResponseEvent.Task && receivedCompatResponseEvent.Task.Result != null)
{
return receivedCompatResponseEvent.Task.Result;
}
else
{
throw new TimeoutException();
}
}

public override void OnEvent(OrchestrationContext context, string name, CompatibilityResponse compatResponse)
{
if (name.Equals("ReceiveCompatResponseEvent")) {
receivedCompatResponseEvent.SetResult(compatResponse);
}
}
}

```
The event is raised from the following endpoint :

```
[HttpPost]
[Route("api/{partitionId}/compatResponse")]
public async Task CompatibilityReportResponse([FromRoute] string partitionId, [FromBody] CompatibilityResponse data)
{
await _workflowClient.Client.RaiseEventAsync(new OrchestrationInstance() { InstanceId = data.OrchestrationInstanceID}, "ReceiveCompatResponseEvent",data);
}
```

The issue is that no matter how long I set the expiration of the timer (1 day, 1 month), whenever I fire an event that triggers this subOrchestration, the timer completes and the subOrchestration fails with TimeoutException();
I was able to work around this by creating a custom activity that simulates "waiting", and then waiting for that task instead of the original timer :

```
public class FakeTimerActivity : TaskActivity>
{
private readonly ILogger _logger;

public FakeTimerActivity(ILogger logger)
{
_logger = logger;
}

protected override Task Execute(TaskContext context, string input)
{
return Task.Delay(50000).ContinueWith(t => "Hello");
}
}

```

And in the orchestration I would use this : ` var timeoutTask = context.ScheduleTask(typeof(FakeTimerActivity));`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.