saltstack / saltstack/salt

Add a fallback value for import_yaml/json/* if the file does not exists

Open
#56,139 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

pending-discussion
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue

I would like to import a list of YAML files but some of them may not exists and the process should not raise an error.

It could be greate to have an option to handle missing file, for example:

{%- import_yaml "missing_file.yaml" as loaded_values default {} %}
Setup

Create a simple test SLS /srv/salt/test.sls:

{% from "test.jinja" import config %}

Test state:
  test.nop:
    - name: {{ config }}

Create the /srv/salt/test.jinja

{%- set config = {} %}

{%- for file in ["file1.yaml", "file2.yaml", "file3.yaml"] %}
  {%- import_yaml file as loaded_values %}
  {%- set config = salt.slsutil.merge(config, loaded_values, strategy='smart') %}
{%- endfor %}

Create the files /srv/salt/file1.yaml:

somekey1:
  subkey1: value1

Create the files /srv/salt/file3.yaml:

somekey3:
  subkey3: value3
Steps to Reproduce Issue

Then try to show the sls:

salt 'minion' state.show_sls test
minion:
    - Rendering SLS 'base:test' failed: Jinja error: file2.yaml
      /var/cache/salt/minion/files/base/test.jinja(4):
      ---
      {%- set config = {} %}
      
      {%- for file in ["file1.yaml", "file2.yaml", "file3.yaml"] %}
        {%- import_yaml file as loaded_values %}    <======================
        {%- set config = salt.slsutil.merge(config, loaded_values, strategy='smart') %}
      {%- endfor %}
      ---
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 392, in render_jinja_tmpl
          output = template.render(**decoded_context)
        File "/usr/lib/python3/dist-packages/jinja2/asyncsupport.py", line 76, in render
          return original_render(self, *args, **kwargs)
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1008, in render
          return self.environment.handle_exception(exc_info, True)
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 780, in handle_exception
          reraise(exc_type, exc_value, tb)
        File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
          raise value.with_traceback(tb)
        File "<template>", line 1, in top-level template code
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1073, in make_module
          return TemplateModule(self, self.new_context(vars, shared, locals))
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1152, in __init__
          body_stream = list(template.root_render_func(context))
        File "/var/cache/salt/minion/files/base/test.jinja", line 4, in top-level template code
          {%- import_yaml file as loaded_values %}
        File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 171, in get_source
          raise TemplateNotFound(template)
      jinja2.exceptions.TemplateNotFound: file2.yaml
      
ERROR: Minions returned with non-zero exit code
Versions Report
Salt Version:
           Salt: 3000
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10
        libgit2: 0.26.0
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.2
         Python: 3.6.9 (default, Nov  7 2019, 10:44:02)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-74-generic
         system: Linux
        version: Ubuntu 18.04 bionic

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the import_yaml handling in salt/utils/jinja.py, using the reported TemplateNotFound path and the example in /srv/salt/test.jinja as the behavioral reference. Done means the shown missing-file import can use its requested default value without failing, while existing imports continue to load normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.