StackStorm / StackStorm/st2

Delay Policy Concurrency = 1

Open
#5,905 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.5k
Forks
787
PR merge metrics
No merged PRs in 30d

Description

Lots of action stuck in schedule state as have delay policy concurrency =1

I calculate the number of action that stuck is around 300+, does this caused because of the delay policy?

Problem statement:
A lot of automation stuck in delayed status, it is an expected behavior as the policy dictate that only one action should run at one time, however we have 300 webhook coming in and this delay the rest 299 and somehow the instance suddenly in hung status, the delay status making all successor action become a zombie action.

Workaround
Rename the action that delayed and rerun.

Stackstorm Version
3.7

Setup & Config
Kubernetes HA
System Requirement is aligned with St2 doc

image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported behavior in StackStorm 3.7 with Kubernetes HA and a delay policy whose concurrency is set to 1. Trace how queued delayed actions affect successor actions and whether the instance reaches hung status; done means the concurrency behavior is understood and delayed actions no longer leave successors as zombies.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.