Azure / Azure/azure-functions-powershell-worker
Wait-DurableTask broken in Powershell 7.2 functions
- Dominant language
- C#
- Stars
- 215
- Forks
- 61
- Avg merge
- 21h 4m
- Merged PRs (30d)
- 6
Description
It seems there is inconsistency and breakage with external event handling when using Powershell 7.2 in Azure functions.
HttpTrigger
```powershell
using namespace System.Net
param($Request, $TriggerMetadata)
$inputs = @{
Name = "$($Request.Query.Name)"
}
$FunctionName = $Request.Params.FunctionName
$InstanceId = Start-DurableOrchestration -FunctionName $FunctionName -Input $inputs
Write-Host "Started orchestration with ID = '$InstanceId'"
$Response = New-DurableOrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
Push-OutputBinding -Name Response -Value $Response
```
Orchestration function
```powershell
using namespace System.Net
param($Context)
$output = @()
$gate1 = Start-DurableExternalEventListener -EventName "Paris" -NoWait -verbose
$gate2 = Start-DurableExternalEventListener -EventName "London" -NoWait -verbose
$output = Invoke-DurableActivity -FunctionName 'Hello' -Input $Context.Input
$endResults = Wait-DurableTask -Task @($gate1, $gate2)
$finaloutput = Invoke-ActivityFunction -FunctionName 'Bye' -Input $output
$finaloutput
```
Hello (Activity)
```powershell
using namespace System.Net
param($name, $TriggerMetadata)
$InstanceId = $TriggerMetadata.InstanceId
Write-Host "Hello $name"
Send-DurableExternalEvent -InstanceId $InstanceId -EventName "London" -verbose
Send-DurableExternalEvent -InstanceId $InstanceId -EventName "Paris" -verbose
$name
```
Bye (Activity)
```powershell
using namespace System.Net
param($name)
Write-Host "Bye $name"
$name
```
Using the above code when I do:
"FUNCTIONS_WORKER_RUNTIME_VERSION": "~7"
It works and will run Bye and produce the correct output. However, when I change it to:
"FUNCTIONS_WORKER_RUNTIME_VERSION": "7.2"
It stays in the running state and does not run Bye. Sending the 2 events manually again, it throws an exception below:
```powershell
Orchestration completed with a 'Failed' status and 314 bytes of output. Details: Unhandled exception while executing orchestration: DurableTask.Core.Exceptions.NonDeterministicOrchestrationException: Non-Deterministic workflow detected: A previous execution of this orchestration scheduled an activity task with sequence ID 1 and name 'Bye' (version ''), but the current replay execution hasn't (yet?) scheduled this task. Was a change made to the orchestrator code after this instance had already started running?
[2022-08-26T06:44:07.415Z] at DurableTask.Core.TaskOrchestrationContext.HandleTaskScheduledEvent(TaskScheduledEvent scheduledEvent) in /_/src/DurableTask.Core/TaskOrchestrationContext.cs:line 271
[2022-08-26T06:44:07.415Z] at DurableTask.Core.TaskOrchestrationExecutor.ProcessEvent(HistoryEvent historyEvent) in /_/src/DurableTask.Core/TaskOrchestrationExecutor.cs:line 189
[2022-08-26T06:44:07.416Z] at DurableTask.Core.TaskOrchestrationExecutor.g__ProcessEvents|11_0(IEnumerable`1 events) in /_/src/DurableTask.Core/TaskOrchestrationExecutor.cs:line 114
[2022-08-26T06:44:07.416Z] at DurableTask.Core.TaskOrchestrationExecutor.ExecuteCore(IEnumerable`1 pastEvents, IEnumerable`1 newEvents) in /_/src/DurableTask.Core/TaskOrchestrationExecutor.cs:line 122
[2022-08-26T06:44:07.417Z] at DurableTask.Core.TaskOrchestrationContext.HandleTaskScheduledEvent(TaskScheduledEvent scheduledEvent) in /_/src/DurableTask.Core/TaskOrchestrationContext.cs:line 271
[2022-08-26T06:44:07.417Z] at DurableTask.Core.TaskOrchestrationExecutor.ProcessEvent(HistoryEvent historyEvent) in /_/src/DurableTask.Core/TaskOrchestrationExecutor.cs:line 189
[2022-08-26T06:44:07.420Z] at DurableTask.Core.TaskOrchestrationExecutor.g__ProcessEvents|11_0(IEnumerable`1 events) in /_/src/DurableTask.Core/TaskOrchestrationExecutor.cs:line 114
[2022-08-26T06:44:07.420Z] at DurableTask.Core.TaskOrchestrationExecutor.ExecuteCore(IEnumerable`1 pastEvents, IEnumerable`1 newEvents) in /_/src/DurableTask.Core/TaskOrchestrationExecutor.cs:line 122
```
If I make the orchestrator slightly complicated:
Orchestration function
```powershell
using namespace System.Net
param($Context)
$output = @()
$gate1 = Start-DurableExternalEventListener -EventName "Paris" -NoWait -verbose
$gate2 = Start-DurableExternalEventListener -EventName "London" -NoWait -verbose
$output = Invoke-DurableActivity -FunctionName 'Hello' -Input $Context.Input
$output1 = Invoke-DurableActivity -FunctionName 'Hello1' -Input $output
$endResults = Wait-DurableTask -Task @($gate1, $gate2)
$finaloutput = Invoke-ActivityFunction -FunctionName 'Bye' -Input $output1
$finaloutput
```
Hello1 (Activity)
```powershell
using namespace System.Net
param($name)
Write-Host "Hello again $name!!"
$name
```
Then it again does not complete unless I resend the events, but this time I sometimes have to send the events multiple times before it fails again (using ~7 still works though). The more complex the orchestration the more inconsistent it seems to get.
Core Tools Version: 4.0.4736 Commit hash: N/A (64-bit)
Function Runtime Version: 4.8.1.18957
When using ~7 it runs PowerShell 7.0.11
When using 7.2 its running PowerShell 7.2.4
Am I doing something wrong?
Anyone else have success running external events with PowerShell 7.2 when using complex orchestrations?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the orchestration with the shown HttpTrigger, orchestration, Hello, and Bye functions under PowerShell 7.0.11 and 7.2.4, comparing FUNCTIONS_WORKER_RUNTIME_VERSION ~7 and 7.2. Inspect the behavior around Start-DurableExternalEventListener and Wait-DurableTask. Done means both external events allow the orchestration to run Bye without repeated event delivery or a non-deterministic workflow exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, powershell
- Domain
- backend, cloud, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100