Template renderers cannot receive results of gpg renderer
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
The GPG renderer is a bit different from most renderers in that it is designed to operate recursively on objects, returning a copy of the object with all the ciphertext decrypted. This works great for a number of applications outside of SLS rendering, but it means that any of the template renderers (jinja, etc.) fail if they are expected to operate on the results of the gpg renderer. For example:
#!gpg|jinja|yaml
-----BEGIN PGP MESSAGE----
.... (ciphertext here) ...
-----END PGP MESSAGE-----
In this case, the gpg renderer will return a string, but this will cause a traceback when it hits the jinja renderer, because from_str is not set. The relevant block of code is here, taken from the head of the 2018.3 branch (note that this issue is also present in all non-EOL Salt releases).
It is not clear to me whether the correct fix is to modify the renderers to pass from_str=True when the input is not file-like, or change the logic in the block of code I linked. I'm not sure what the reason is for from_str, and what edge case this is fixing, and don't want to make a fix that will break other code in the process. Any feedback is appreciated.
Steps to reproduce
Setup
>>> import salt.loader
>>> import salt.config
>>> opts = salt.config.minion_config('/etc/salt/minion')
opts>>> opts['file_client'] = 'local'
>>> rend = salt.loader.render(opts, {'test.ping': lambda: True})
>>> template_data = "{{ salt['test.ping']() }}"
Simulate passing as a string (as in what happens with GPG renderer)
>>> result = rend['jinja'](template_data)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/testing/salt/renderers/jinja.py", line 70, in render
**kws)
File "/testing/salt/utils/templates.py", line 165, in render_tmpl
six.reraise(*sys.exc_info())
File "/testing/salt/utils/templates.py", line 151, in render_tmpl
with codecs.open(tmplsrc, 'r', SLS_ENCODING) as _tmplsrc:
File "/usr/lib64/python2.7/codecs.py", line 881, in open
file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: "{{ salt['test.ping']() }}"
Simulating passing data in as a StringIO (works as expected)
>>> from salt.ext.six.moves import StringIO
>>> result = rend['jinja'](StringIO(template_data))
>>> result.read()
u'True'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with salt/utils/templates.py around lines 144-152 and the jinja renderer in renderers/jinja.py, then reproduce the failure by passing a string to the loaded jinja renderer as shown. Compare that with the working StringIO case; done means a gpg|jinja|yaml renderer chain accepts decrypted string output without the traceback.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100