saltstack / saltstack/salt

[BUG] Single quotation mark in contents_pillar not escaped when using {% raw %}

Open
#66,138 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Single quotation marks in a {% raw %} block inserted via contents_pillar are not escaped properly. See example below.

Setup
Pillar:

one:
  instances:
    - name: three
      well_known_acme_challenge: True
      files:
        fileone.json: |
          {
              "MY_VALUE": "value"
          }
        # The following YAML pillar contents has Jinja that should be ignored by Salt,
        # so I am using raw. This is a config file for another application.
        filetwo.yaml: |
          {% raw %}
          # ---
          # This file is managed by Saltstack on this machine. Manual modifications
          # to this file may be overridden when states are next applied.
          # ---
          somevalue:
            #   This comment's problem is it creates an error
            - anothervalue: >-
                {{ somejinjaexpression and
                  anotherjinjaexpression }}
           {% endraw %}

Salt:

{% set instance_index0 = loop.index0 %}
{% if (salt['pillar.get']('one:instances:' + (instance_index0 | string) + ':files', default={}) | length) > 0 %}
one_instances_{{instance.name}}_config:
  file.managed:
    - makedirs: True
    - names:
      {%- for file in salt['pillar.get']('one:instances:' + (instance_index0 | string) + ':files', default={}) %}
      - /path/to/{{instance.name}}/{{ file }}:
        # The following contents_pillar cannot be surrounded by raw because it relies
        # on templates for the value
        - contents_pillar: one:instances:{{ instance_index0 }}:files:{{ file }}
        - user: root
        - group: root
        - mode: '0640'
      {%- endfor %}
{%- endif %}

Error:

ERROR: Minions returned with non-zero exit code
myhost.example.com:
    Data failed to compile:
----------
    Rendering SLS 'base:app.one' failed: while parsing a flow mapping
  in "<unicode string>", line 28, column 160:
     ... acme_challenge': True, 'files': {'fileone.json': '{\n    "MY_VAL ... 
                                         ^
expected ',' or '}', but got '<scalar>'
  in "<unicode string>", line 28, column 2651:
     ... \n  #   This comment\'s problem is it creates an error ... 
                               ^

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM on KVM (TrueNAS SCALE)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

Steps to Reproduce the behavior
See above configs.

Expected behavior
Should insert file without error.

Screenshots
N/A, see console output above.

Versions Report

salt --versions-report
Salt Version:
          Salt: 3006.6
 
Python Version:
        Python: 3.10.13 (main, Nov 15 2023, 04:34:27) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.3
       libgit2: 1.7.2
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: 1.14.1
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.14.2
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 12 bookworm
        locale: utf-8
       machine: x86_64
       release: 6.1.0-18-amd64
        system: Linux
       version: Debian GNU/Linux 12 bookworm

Additional context
Possibly an additional option like contents_pillar_raw would help?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied Salt and pillar configurations, focusing on the contents_pillar path with a {% raw %} block containing a single quotation mark. Trace how that content is rendered and parsed, then verify that the file is inserted without a YAML parsing error while normal templating still works.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.