StackStorm / StackStorm/st2

Ability to tell Orquesta to treat a variable as raw string and not do any (YAQL and Jinja) processing on it

Open
#4,636 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement workflows: orquesta
Dominant language
Python
Stars
6.5k
Forks
787
PR merge metrics
No merged PRs in 30d

Description

I was debugging an Orquesta flow and I initially thought it was a bug in Orquesta (due to the error message), but it turns out that the bug is related to dynamically retrieved data containing a YAQL expression.

It's fairly common scenario that data inside workflows is retrieved dynamically and this data could potentially contain Jinja or YAQL escape sequence (think Jinja / YAQL injection).

Here is an example workflow where this issue can arise - https://github.com/StackStorm/stackstorm-st2community/blob/master/actions/workflows/retrieve_data_and_send_daily_stats_to_slack.yaml

github_data and / or forum_posts can contain YAQL and Jinja expression and assemble_message task would fail publishing message variable since it would try to render it as YAQL / Jinja and this would fail due to variable not being defined - https://github.com/StackStorm/stackstorm-st2community/blob/master/actions/workflows/retrieve_data_and_send_daily_stats_to_slack.yaml#L46.

Yes, I could process / escape this data inside st2community.assemble_message (and that's what I will do until a better solution is in place), but I think we should provide an Orquesta level feature for handling that.

Pretty much all the programming languages and templating libraries provide some kind of way to treat a value as raw string / escape it and not do any processing on it and I think that's what we need in Orquesta as well (ability to mark a variable as raw so it's not processed).

I also think that's important from security perspective because there are a lot of scenarios where data is retrieved dynamically or passed in by a user.

I'm also open to suggestions (aka if this can already be solved in some other way).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Use the referenced stackstorm-st2community/actions/workflows/retrieve_data_and_send_daily_stats_to_slack.yaml workflow as the entry point, especially the assemble_message task and its message variable. Trace how dynamically retrieved values reach YAQL and Jinja processing; completion would be a defined raw-string mechanism that prevents both renderers from evaluating marked values, with coverage for the example behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.