ActionChain runner fails to load default parameter of action from pack configuration

SUMMARY

In an ActionChain, when calling a Mistral workflow that has a default parameter defined from the pack configuration, the parameter default is not loaded from configuration, resulting in an error when trying to run the Mistral workflow. It appears like the Jinja expression defining the default parameter is not being rendered:

error: Failed running task "example_task".. u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'

STACKSTORM VERSION

st2 2.8.1, on Python 2.7.6

OS, environment, install method

StackStorm Docker on macOS.

Steps to reproduce the problem

First, a Mistral workflow with a default parameter from the pack configuration:

---
name: test_config_default
pack: ers
runner_type: mistral-v2
description: test
enabled: true
entry_point: workflows/test_config_default.yaml
parameters:
  hosts:
    type: array
    required: true
    default: "{{ config_context.install_pack.hosts }}"

---
version: '2.0'

ers.test_config_default:
  description: test
  type: direct
  input:
    - hosts

  tasks:
    print:
      action: core.echo
      input:
        message: "{{ _.hosts | join(',') }}"

Now, an ActionChain that calls this Mistral workflow:

---
name: test_action_chain
pack: ers
runner_type: action-chain
description: test
enabled: true
entry_point: chains/test_action_chain.yaml

---
chain:
  - name: example_task
    ref: ers.test_config_default

Run the test_config_default Mistral workflow:

root@59ce30ee29dd:/# st2 run ers.test_config_default
.
id: 5daa170e53b5c808202e8fe0
action.ref: ers.test_config_default
parameters: 
  hosts:
  - 10.42.43.247
  - 10.42.43.217
  - 10.42.43.233
  - 10.42.43.234
  - 10.42.43.245
  - 10.42.43.240
status: succeeded
result_task: print
result: 
  failed: false
  return_code: 0
  stderr: ''
  stdout: 10.42.43.247,10.42.43.217,10.42.43.233,10.42.43.234,10.42.43.245,10.42.43.240
  succeeded: true
start_timestamp: Fri, 18 Oct 2019 19:48:30 UTC
end_timestamp: Fri, 18 Oct 2019 19:48:31 UTC

Great, the default param was loaded from the pack config. But now run the test_action_chain:

root@59ce30ee29dd:/# st2 run ers.test_action_chain
.
id: 5daa175553b5c808202e8fe5
action.ref: ers.test_action_chain
parameters: None
status: failed
result_task: example_task
result: 
  error: "Failed running task "example_task".. u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'"
  traceback: "Traceback (most recent call last):
  File "/opt/stackstorm/runners/action_chain_runner/action_chain_runner/action_chain_runner.py", line 516, in _run_chain
    liveaction = self._run_action(liveaction)
  File "/opt/stackstorm/runners/action_chain_runner/action_chain_runner/action_chain_runner.py", line 734, in _run_action
    raise e
ValidationError: u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'
"
error: Failed running task "example_task".. u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'
traceback: Traceback (most recent call last):
  File "/opt/stackstorm/runners/action_chain_runner/action_chain_runner/action_chain_runner.py", line 516, in _run_chain
    liveaction = self._run_action(liveaction)
  File "/opt/stackstorm/runners/action_chain_runner/action_chain_runner/action_chain_runner.py", line 734, in _run_action
    raise e
ValidationError: u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'

failed_on: example_task
start_timestamp: Fri, 18 Oct 2019 19:49:41 UTC
end_timestamp: Fri, 18 Oct 2019 19:49:42 UTC
result: 
  error: "Failed running task "example_task".. u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'"
  traceback: "Traceback (most recent call last):
  File "/opt/stackstorm/runners/action_chain_runner/action_chain_runner/action_chain_runner.py", line 516, in _run_chain
    liveaction = self._run_action(liveaction)
  File "/opt/stackstorm/runners/action_chain_runner/action_chain_runner/action_chain_runner.py", line 734, in _run_action
    raise e
ValidationError: u'{{ config_context.install_pack.hosts }}' is not of type u'array'

Failed validating u'type' in schema['properties'][u'hosts']:
    {u'default': u'{{ config_context.install_pack.hosts }}',
     u'required': True,
     u'type': u'array'}

On instance[u'hosts']:
    u'{{ config_context.install_pack.hosts }}'
"

Again, appears as the the Jinja expression is not even being evaluated before being passed as an input to the test_config_default action.

Notably, the test_config_default action (with the param default from config) can be called from a Mistral workflow:

---
name: test_mistral
pack: ers
runner_type: mistral-v2
description: test
enabled: true
entry_point: workflows/test_mistral.yaml

---
version: '2.0'

ers.test_mistral:
  description: test
  type: direct

  tasks:
    test:
      action: ers.test_config_default

Run test_mistral:

root@59ce30ee29dd:/# st2 run ers.test_mistral
..
id: 5daa17fb53b5c808202e8fe8
action.ref: ers.test_mistral
parameters: None
status: succeeded
result_task: test
result: 
  extra:
    state: SUCCESS
    state_info: null
  tasks:
  - created_at: '2019-10-18 19:52:27'
    id: 7b8d7f23-82dc-4b7e-882e-5181090c7cd6
    input: null
    name: print
    published: {}
    result:
      failed: false
      return_code: 0
      stderr: ''
      stdout: 10.42.43.247,10.42.43.217,10.42.43.233,10.42.43.234,10.42.43.245,10.42.43.240
      succeeded: true
    state: SUCCESS
    state_info: null
    updated_at: '2019-10-18 19:52:28'
    workflow_execution_id: 27a647c0-4afb-4d10-82f6-f84bc98cd496
    workflow_name: ers.test_config_default
start_timestamp: Fri, 18 Oct 2019 19:52:27 UTC
end_timestamp: Fri, 18 Oct 2019 19:52:30 UTC
+-----------------------------+------------------------+-------+-------------------------+--------------------------+
| id                          | status                 | task  | action                  | start_timestamp          |
+-----------------------------+------------------------+-------+-------------------------+--------------------------+
| + 5daa17fb53b5c808202e8feb  | succeeded (2s elapsed) | test  | ers.test_config_default | Fri, 18 Oct 2019         |
|                             |                        |       |                         | 19:52:27 UTC             |
|    5daa17fc53b5c808202e8fed | succeeded (1s elapsed) | print | core.echo               | Fri, 18 Oct 2019         |
|                             |                        |       |                         | 19:52:27 UTC             |
+-----------------------------+------------------------+-------+-------------------------+--------------------------+

Expected Results

I would expect calling test_config_default from the ActionChain above to be successful, as it is when calling it from a Mistral workflow.

Actual Results

Calling test_config_default from an ActionChain does not work. Calling it from a Mistral workflow does work.

Start in action_chain_runner/action_chain_runner/action_chain_runner.py at the _run_chain and _run_action locations shown in the traceback. Reproduce the ActionChain and compare it with the working direct and Mistral workflow calls, focusing on when the pack-configuration default is rendered. Done means the ActionChain passes the resolved hosts array to the Mistral action and completes successfully.