StackStorm / StackStorm/st2

Orquesta throws python exception if core.ask is run in parallel another task

Open
#4,864 5 comments 1 reaction 1 assignee View on GitHub

@m4dcoder is already working on this.

Since Feb 12, 2020.

bug inquiries status:to be verified workflows: orquesta
Dominant language
Python
Stars
6.5k
Forks
787
PR merge metrics
No merged PRs in 30d

Description

SUMMARY

Orquesta can't handle pausing for core.ask while a parallel task is running

STACKSTORM VERSION

st2 3.1.0, on Python 2.7.5

OS, environment, install method

RHEL 7.6, script install

Steps to reproduce the problem

Create a workflow that starts two tasks in parallel, one of which is a core.ask inquiry
Run the workflow and answer the inquiry while the parallel task is running
Use
st2 execution get
after answering the inquiry, but before the parallel task has completed

Example Orquesta workflow: https://gist.github.com/markcrobinson/1618c90d2c2b66f0ae46a64c25b132d6

Expected Results

The workflow allows you to answer the inquiry and then completes successfully

Actual Results

A python error such as:

$  st2 execution get 5e43e5da103007aab0b8bcf2
id: 5e43e5da103007aab0b8bcf2
action.ref: testing.test-parallel-pause
parameters: None
status: failed (16s elapsed)
start_timestamp: Wed, 12 Feb 2020 11:47:38 UTC
end_timestamp: Wed, 12 Feb 2020 11:47:54 UTC
result:
  error: Unable to process event "workflow_paused" for workflow in "pausing" status.
  traceback: "  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2actions/container/base.py", line 193, in _do_pause
    (status, result, context) = runner.pause()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/orquesta_runner/orquesta_runner.py", line 136, in pause
    wf_ex_db = wf_svc.request_pause(self.execution)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/services/workflows.py", line 312, in request_pause
    conductor.request_workflow_status(statuses.PAUSED)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/orquesta/conducting.py", line 363, in request_workflow_status
    raise exc.InvalidWorkflowStatusTransition(current_status, wf_ex_event.name)
"
+--------------------------+-------------------------+------------+------------+-------------------------------+
| id                       | status                  | task       | action     | start_timestamp               |
+--------------------------+-------------------------+------------+------------+-------------------------------+
| 5e43e5db103007aaf8abdee5 | succeeded (0s elapsed)  | start      | core.echo  | Wed, 12 Feb 2020 11:47:39 UTC |
| 5e43e5dc103007aaf8abdee8 | succeeded (14s elapsed) | do_inquiry | core.ask   | Wed, 12 Feb 2020 11:47:40 UTC |
| 5e43e5dc103007aaf8abdeeb | running (17s elapsed)   | snooze     | core.local | Wed, 12 Feb 2020 11:47:40 UTC |
+--------------------------+-------------------------+------------+------------+-------------------------------+

This exception is cleared once the parallel task is done, but the execution stays in paused state:

$  st2 execution get 5e43e5da103007aab0b8bcf2
id: 5e43e5da103007aab0b8bcf2
action.ref: testing.test-parallel-pause
parameters: None
status: paused
start_timestamp: Wed, 12 Feb 2020 11:47:38 UTC
end_timestamp: Wed, 12 Feb 2020 11:47:54 UTC
result:
  output: null
+--------------------------+-------------------------+------------+------------+-------------------------------+
| id                       | status                  | task       | action     | start_timestamp               |
+--------------------------+-------------------------+------------+------------+-------------------------------+
| 5e43e5db103007aaf8abdee5 | succeeded (0s elapsed)  | start      | core.echo  | Wed, 12 Feb 2020 11:47:39 UTC |
| 5e43e5dc103007aaf8abdee8 | succeeded (14s elapsed) | do_inquiry | core.ask   | Wed, 12 Feb 2020 11:47:40 UTC |
| 5e43e5dc103007aaf8abdeeb | succeeded (60s elapsed) | snooze     | core.local | Wed, 12 Feb 2020 11:47:40 UTC |
+--------------------------+-------------------------+------------+------------+-------------------------------+

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.