workflows run in unit test executor do not behave the same way as when running normally and give opaque errors.
- Dominant language
- TypeScript
- Stars
- 111
- Forks
- 109
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 20
Description
### Severity
P3 - Medium (Minor functionality affected)
### Describe the Bug with repro steps
The workflow definition given below returns the following result when executed
If you click the "Create unit test from run" button and run the created unit tests there is an unhandled exception in the
`RunWorkflowAsync` method of the `TestExecutor`
The error message is similar to the below
```
Failed Stateful1_ErrorDemo_ExecuteWorkflow_SUCCESS_Sample1 [15 s]
Error Message:
Test method LogicApps.Tests.ErrorDemo.Stateful1_ErrorDemo_ExecuteWorkflow_SUCCESS_Sample1 threw exception:
System.ArgumentNullException: Value cannot be null. (Parameter 'value')
Stack Trace:
at Microsoft.Azure.Workflows.UnitTesting.TestHost.UnitTestEngine.ExecuteWorkflow(FlowPropertiesDefinition workflowDefinition, TriggerMock triggerMock, Dictionary`2 actionMocks, String appDirectory, String connections, String parameters, IDictionary`2 localSettings, String customCodeFunctionFilePath, Int32 timeoutInSeconds)
at Microsoft.Azure.Workflows.UnitTesting.TestHost.UnitTestEngine.ExecuteWorkflow(FlowPropertiesDefinition workflowDefinition, TriggerMock triggerMock, Dictionary`2 actionMocks, String appDirectory, String connections, String parameters, IDictionary`2 localSettings, String customCodeFunctionFilePath, Int32 timeoutInSeconds)
at Microsoft.Azure.Workflows.UnitTesting.UnitTestExecutor.RunWorkflowAsync(TestMockDefinition testMock, Int32 timeoutInSeconds)
at LogicApps.Tests.ErrorDemo.Stateful1_ErrorDemo_ExecuteWorkflow_SUCCESS_Sample1() in ...\LogicAppsWorkspace\Tests\LogicApps\Stateful1\ErrorDemo\ErrorDemo.cs:line 47
```
The expected result would be that the test executor should behave the same way as the product does. It should run the test execution successfully to conclusion and at the end of it we should see that the Compose/Scope have status of failed and that "Error Response" was executed.
If it was to return an unhandled error then it would at least be useful if this had some correlation to what the UI shows for this case "Failed to evaluate the tracked properties for action 'Compose'. Error details: 'The template language expression 'action().outputs.foo.bar' cannot be evaluated because property 'foo' doesn't exist, available properties are 'functionResult, timestamp'. Please see https://aka.ms/logicexpressions for usage details.'."
For the simple workflow in this example it is clear what the issue is but this type of error can be difficult to debug in more complicated workflows (where the issue might arise in some path not frequently encountered in normal executions or may just be an issue with the mocked data in the test).
### What type of Logic App Is this happening in?
Standard (VSCode)
### Are you experiencing a regression?
_No response_
### Which operating system are you using?
Windows
### Did you refer to the TSG before filing this issue? https://aka.ms/lauxtsg
Yes
### Workflow JSON
```json
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Scope": {
"type": "Scope",
"actions": {
"Call_a_local_function_in_this_logic_app": {
"type": "InvokeFunction",
"inputs": {
"functionName": "FunctionApp",
"parameters": {
"zipCode": 85396,
"temperatureScale": "Celsius"
}
}
},
"Compose": {
"type": "Compose",
"inputs": {
"functionResult": "@body('Call_a_local_function_in_this_logic_app')",
"timestamp": "@utcNow()"
},
"runAfter": {
"Call_a_local_function_in_this_logic_app": [
"SUCCEEDED"
]
},
"trackedProperties": {
"causeError": "@action().outputs.foo.bar"
}
}
},
"runAfter": {}
},
"Response": {
"type": "Response",
"kind": "http",
"inputs": {
"statusCode": 200,
"body": "@body('Call_a_local_function_in_this_logic_app')"
},
"runAfter": {
"Scope": [
"SUCCEEDED"
]
}
},
"Error_Response": {
"type": "Response",
"kind": "Http",
"inputs": {
"statusCode": 400
},
"runAfter": {
"Scope": [
"FAILED"
]
}
}
},
"triggers": {
"When_a_HTTP_request_is_received": {
"type": "Request",
"kind": "Http"
}
},
"contentVersion": "1.0.0.0",
"outputs": {}
},
"kind": "Stateful"
}
```
### Screenshots or Videos
_No response_
### Environment
-
### Additional context
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.