elsa-workflows / elsa-workflows/elsa-core

Concurrency problems saving and publishing workflow definitions

Open
#3,179 0 comments 2 reactions 0 assignees View on GitHub
bug triaged
Dominant language
C#
Stars
7.9k
Forks
1.5k
Avg merge
15h 22m
Merged PRs (30d)
114

Description

Investigating #3154 I found that Elsa behaves unpredictably if multiple save/publish request are sent for the same workflowDefinitionId concurrently.

This error was first produced on the dashboard UI during normal usage.
For the purpose of consistent reproduction I made a small node.js script that you can find in the following gist.

```
git clone https://gist.github.com/GotBinGo/dd6b292905b632bc3834cd1d3ddf5757 elsa-concurrent-test
cd elsa-concurrent-test
npm i
```

You can run the example with the following command.
```
node elsa-concurrent-test.js
```

The main part is the following.

```
await createWorkflow(aok);

await Promise.all([
createWorkflow(aok),
publishById('7c1d3a98f255441cbe085056f1351b04')
])

await Promise.all([
publishById('7c1d3a98f255441cbe085056f1351b04'),
createWorkflow(bok),
]);
```

In this example I have two versions(aok, bok) of the same workflow.
The steps look something like this.

1. Save version A.
2. Concurrently save version A and publish it.
3. Concurrently save version B and publish it.

**Sometimes** this results in Workflow Definitions List showing the latest workflow version 2.
The list also shows that there is no published version of the workflow.
That is strange, as version 1 was published, and never unpublished.

Most of our problem is that after this, the **workflow definition is stuck** in a state where saving/updating the workflow then publishing it does not work as intended, there is no way to access any version of the endpoint in this workflow definition.
After restarting the instance everything is back to normal.
Deleting the workflow and recreating the same one under a different workflowDefinitionId fixes the problem too.

I was able to consistently reproduce this stuck state with the following setups.
- Our custom Multinode setup
- **Official single node docker setup**
You can just run the [docker](https://elsa-workflows.github.io/elsa-core/docs/quickstarts/quickstarts-elsa-and-docker) command from here and run the script I just linked.

If you don't get the stuck state on the first try, just try deleting the created workflow and running the script again.
In my experience the more load(higher response time) you have on Elsa the more likely you are to see this error.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.