saltstack / saltstack/salt

Template renderers cannot receive results of gpg renderer

Open
#54,200 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Confirmed severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

The GPG renderer is a bit different from most renderers in that it is designed to operate recursively on objects, returning a copy of the object with all the ciphertext decrypted. This works great for a number of applications outside of SLS rendering, but it means that any of the template renderers (jinja, etc.) fail if they are expected to operate on the results of the gpg renderer. For example:

#!gpg|jinja|yaml
-----BEGIN PGP MESSAGE----

.... (ciphertext here) ...
-----END PGP MESSAGE-----

In this case, the gpg renderer will return a string, but this will cause a traceback when it hits the jinja renderer, because from_str is not set. The relevant block of code is here, taken from the head of the 2018.3 branch (note that this issue is also present in all non-EOL Salt releases).

It is not clear to me whether the correct fix is to modify the renderers to pass from_str=True when the input is not file-like, or change the logic in the block of code I linked. I'm not sure what the reason is for from_str, and what edge case this is fixing, and don't want to make a fix that will break other code in the process. Any feedback is appreciated.

Steps to reproduce
Setup
>>> import salt.loader
>>> import salt.config
>>> opts = salt.config.minion_config('/etc/salt/minion')
opts>>> opts['file_client'] = 'local'
>>> rend = salt.loader.render(opts, {'test.ping': lambda: True})
>>> template_data = "{{ salt['test.ping']() }}"
Simulate passing as a string (as in what happens with GPG renderer)
>>> result = rend['jinja'](template_data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/testing/salt/renderers/jinja.py", line 70, in render
    **kws)
  File "/testing/salt/utils/templates.py", line 165, in render_tmpl
    six.reraise(*sys.exc_info())
  File "/testing/salt/utils/templates.py", line 151, in render_tmpl
    with codecs.open(tmplsrc, 'r', SLS_ENCODING) as _tmplsrc:
  File "/usr/lib64/python2.7/codecs.py", line 881, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: "{{ salt['test.ping']() }}"
Simulating passing data in as a StringIO (works as expected)
>>> from salt.ext.six.moves import StringIO
>>> result = rend['jinja'](StringIO(template_data))
>>> result.read()
u'True'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with salt/utils/templates.py around lines 144-152 and the jinja renderer in renderers/jinja.py, then reproduce the failure by passing a string to the loaded jinja renderer as shown. Compare that with the working StringIO case; done means a gpg|jinja|yaml renderer chain accepts decrypted string output without the traceback.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.