StackStorm / StackStorm/st2

Configs not loading lists from kv store

Open
#4,953 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.5k
Forks
787
PR merge metrics
No merged PRs in 30d

Description

SUMMARY

When registering a config with a list parameter that is referenced from the Key Value store, it is not being loaded properly which makes the register-configs command fail.

STACKSTORM VERSION
# st2 --version
st2 3.2.0, on Python 2.7.5
OS, environment, install method
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.8 (Maipo)

Installed with puppet module

Problem

Config schema: https://github.com/EncoreTechnologies/stackstorm-patching_roulette/blob/master/config.schema.yaml

We have the Key Value store values:

st2 key get patching_roulette.members
+------------------+--------------------------------+
| Property         | Value                          |
+------------------+--------------------------------+
| name             | patching_roulette.members      |
| value            | ["test_user_1", "test_user_2"] |
| secret           | False                          |
| encrypted        | False                          |
| scope            | st2kv.system                   |
| expire_timestamp |                                |
+------------------+--------------------------------+

# st2 key get patching_roulette.assignments
+------------------+--------------------------------------------+
| Property         | Value                                      |
+------------------+--------------------------------------------+
| name             | patching_roulette.assignments              |
| value            | ["test_assignment_1", "test_assignment_2"] |
| secret           | False                                      |
| encrypted        | False                                      |
| scope            | st2kv.system                               |
| expire_timestamp |                                            |
+------------------+--------------------------------------------+

The config:

# cat /opt/stackstorm/configs/patching_roulette.yaml 
---
members: "{{ st2kv.system.patching_roulette.members }}"
assignments: "{{ st2kv.system.patching_roulette.assignments }}"

And when we try to register it produces this error:

 st2ctl reload --register-configs
Registering content...[flags = --config-file /etc/st2/st2.conf --register-configs]
2020-05-18 11:35:03,345 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "stackstorm".
2020-05-18 11:35:03,353 INFO [-] Successfully connected to database "st2" @ "127.0.0.1:27017" as user "stackstorm".
2020-05-18 11:35:04,053 INFO [-] =========================================================
2020-05-18 11:35:04,053 INFO [-] ############## Registering configs ######################
2020-05-18 11:35:04,054 INFO [-] =========================================================
2020-05-18 11:35:05,366 WARNING [-] Failed to register configs: Failed to register config "/opt/stackstorm/configs/patching_roulette.yaml" for pack "patching_roulette": Failed validating attribute "assignments" in config for pack "patching_roulette" (/opt/stackstorm/configs/patching_roulette.yaml): '{{ st2kv.system.patching_roulette.assignments }}' is not of type u'array', 'null'

Failed validating u'type' in schema['properties'][u'assignments']:
    {u'description': u'List of patching assignments',
     u'items': {u'type': u'string'},
     u'type': [u'array', 'null']}

On instance[u'assignments']:
    '{{ st2kv.system.patching_roulette.assignments }}'
Traceback (most recent call last):
  File "/usr/bin/st2-register-content", line 22, in <module>
    sys.exit(content_loader.main(sys.argv[1:]))
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 407, in main
    register_content()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 390, in register_content
    register_configs()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 345, in register_configs
    raise e
ValueError: Failed to register config "/opt/stackstorm/configs/patching_roulette.yaml" for pack "patching_roulette": Failed validating attribute "assignments" in config for pack "patching_roulette" (/opt/stackstorm/configs/patching_roulette.yaml): '{{ st2kv.system.patching_roulette.assignments }}' is not of type u'array', 'null'

Failed validating u'type' in schema['properties'][u'assignments']:
    {u'description': u'List of patching assignments',
     u'items': {u'type': u'string'},
     u'type': [u'array', 'null']}

On instance[u'assignments']:
    '{{ st2kv.system.patching_roulette.assignments }}'

When we change the config to not use the values from the Key Value store:

# cat /opt/stackstorm/configs/patching_roulette.yaml 
---
members: 
  - "test_user_1"
  - "test_user_2"
assignments: 
  - "test_assignment_1"
  - "test_assignment_2"

Everything works as expected:

# st2ctl reload --register-configs
Registering content...[flags = --config-file /etc/st2/st2.conf --register-configs]
2020-05-18 11:45:16,522 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "stackstorm".
2020-05-18 11:45:16,531 INFO [-] Successfully connected to database "st2" @ "127.0.0.1:27017" as user "stackstorm".
2020-05-18 11:45:17,418 INFO [-] =========================================================
2020-05-18 11:45:17,418 INFO [-] ############## Registering configs ######################
2020-05-18 11:45:17,418 INFO [-] =========================================================
2020-05-18 11:45:19,064 INFO [-] Registered 8 configs.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with st2ctl reload --register-configs using the shown YAML and Key Value entries. Start in st2common/content/bootstrap.py at register_configs and compare the referenced list values with the config schema validation; done means list-valued Key Value entries load successfully and the config registers without the array-type error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.