Passing user context around to rules, sensors and actions

Context

User scoped variables are a new feature in st2. Currently, users can refer to an item in key value store using {{system.foo}} syntax. We want to support {{user.foo}}. The context of who the user is straight forward when actions are run manually (via CLI or UI) but not so obvious when automations are kicked off via rules. Read further.

Related: https://github.com/StackStorm/st2/pull/2669

What?

The problem is pretty simple - User scoped variables cannot be used anywhere in content if the event entering the StackStorm system is not kicked off manually by the user (CLI or UI command to run an action/workflow).

Rules is a good example. There is no notion of "user" when a rule is enforced. The rule is enforced by the "system". This means rules cannot use user scoped variables. As an extension, actions/workflows on the south side of rules can't access them either. We want to address that.

Why?

If we allowed user scoped variables only as values to action parameters when run manually, we will have a serious UX problem. Users will be confused where they can use user scoped variables and where they cannot.

How?

@manasdk, @dzimine and I sat together and discussed some options. To solve the problem, we need to create a concept of "user context".

@manasdk had a suggestion that we can create a "system_user" concept (usually stanley). stanley will need to be valid entry with valid credentials PAM/LDAP/Random_auth_provider. So when people use {{user.foo}} in rule, we'll resolve that to scope=user name=stanley.foo and use the corresponding value from key value store.

I wanted to solve it a similar way but I also want an ability to specify explicitly the user context as a field in rules. So in my model, the rule will look like

---
name: "notify"
pack: "chatops"
enabled: true
description: "Notification rule to send results of action executions to stream for chatops"
trigger:
  type: "core.st2.generic.notifytrigger"
criteria:
  trigger.route:
    pattern: "hubot"
    type: "equals"
action:
  ref: chatops.post_result
  parameters:
    channel: "{{trigger.data.source_channel}}"
    user: "{{trigger.data.user}}"
    execution_id: "{{trigger.execution_id}}"
context:
    run_as: lakshmi  # Should be an authenticatable user in st2 land

The concept of run_as opens up an interesting problem while providing more flexibility. RBAC is a problem that we need to address if we introduced run_as. A rule owner cannot specify an arbitrary user as run_as (unless they are admin). So a rule author can only name themselves as run_as (provided they already have permissions to create the rule.). So in theory this should be possible to implement, st2ctl poses another problem. Today we let anyone register content. This means we lose the idea of who owns the rule. We don't have this problem when authoring rules via API. One potential option is to disable st2ctl from being accessed by non-admins via file permissions so regular users would always use the API thereby RBAC can be enforced.

@dzimine wants more (like usual!). My idea was that rules being the connecting link between triggers and actions, it was a good place to throw the user context. One other reason is that the incoming events and outgoing actions can work with entirely different auth systems which don't have a corresponding mapping in st2 auth land. For example, AWS users need not mirror LDAP users one-to-one. You can say the same thing about incoming triggers (new relic or nagios or ceilometer). So dzimine wants to define the user context based on incoming triggers as well. In his mental model, an incoming trigger would call out the user (optional). Rule can now specify run_as user as a dynamic user. An example rule:

---
name: "notify"
pack: "chatops"
enabled: true
description: "Notification rule to send results of action executions to stream for chatops"
trigger:
  type: "core.st2.generic.notifytrigger"
criteria:
  trigger.route:
    pattern: "hubot"
    type: "equals"
action:
  ref: chatops.post_result
  parameters:
    channel: "{{trigger.data.source_channel}}"
    user: "{{trigger.data.user}}"
    execution_id: "{{trigger.execution_id}}"
context:
    run_as: {{trigger.user}}  # XXX: ``user`` should have a valid mapping in st2 auth land

The problem with doing this is that we need to maintain a map of this external system user to an authenticatable user in st2 land. The mapping will be provided by admin via a YAML. Note that this just means that we can use the variables of the st2 user. It DOES NOT mean you can run the action or workflow as that user. So I don't think this solution is complete. I see the need for two users 1. Incoming triggers with some user 2. Outgoing actions with some user. Also note that there needs to be mapping of those different users to an actual authenticatable st2 user. To give an example, see the rule below:

---
name: "notify"
pack: "chatops"
enabled: true
description: "Notification rule to send results of action executions to stream for chatops"
trigger:
  type: "core.st2.generic.notifytrigger"
criteria:
  trigger.route:
    pattern: "hubot"
    type: "equals"
action:
  ref: chatops.post_result
  parameters:
    channel: "{{trigger.data.source_channel}}"
    user: "{{trigger.data.user}}"
    execution_id: "{{trigger.execution_id}}"
context:
    trigger_user: "{{input_user_map.get(trigger.user)}}"  # Resultant should have a valid mapping in st2 auth land
    action_user: "{{input_output_user_maps.get(trigger.user)}}" # Resultant user should be a valid mapping in st2 auth land. 

Please don't focus on the jinja syntax but rather on the idea.

Start by reviewing related PR #2669 and the rule, trigger, and action examples in this issue; no source files or tests are named. The work needs an agreed user-context design covering run_as, incoming and outgoing users, authentication mapping, and RBAC before implementation can be considered done.