saltstack / saltstack/salt

salt URL with unsupported characters for windows path should be mangled on the minion file system

Open
#58,726 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug File Servers severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description

I'm building salt:// URLs with colons in it to represent the grains related files I'm loading and this fails on windows.

The URL salt://dns:domain/example.net.yaml is mangled as salt://dns_domain/example.net.yaml and the minion can't find that file:

2020-10-13 19:08:36,086 [salt.fileclient  :1129][DEBUG   ][4228] Could not find file 'salt://dns_domain/example.net.yaml' in saltenv 'base'

If I put a symlink on the fileserver:

ln -s /srv/salt/dns:domain /srv/salt/dns_domain

then, the windows minion is failing for another reason:

2020-10-13 19:13:18,382 [salt.fileclient  :1152][DEBUG   ][1700] In saltenv 'base', looking at rel_path 'dns_domain/example.net.yaml' to resolve 'salt://dns_domain/example.net.yaml'
2020-10-13 19:13:18,382 [salt.fileclient  :1161][DEBUG   ][1700] In saltenv 'base', ** considering ** path 'c:\salt\var\cache\salt\minion\files\base\dns_domain\example.net.yaml' to resolve 'salt://dns_domain/example.net.yaml'
2020-10-13 19:13:18,382 [salt.fileclient  :1179][DEBUG   ][1700] Fetching file from saltenv 'base', ** attempting ** 'salt://dns_domain/example.net.yaml'
2020-10-13 19:13:18,382 [salt.fileclient  :1207][DEBUG   ][1700] No dest file found
2020-10-13 19:13:18,398 [salt.fileclient  :1295][INFO    ][1700] Fetching file from saltenv 'base', ** done ** 'dns_domain/example.net.yaml'

It looks like the minion is mangling the salt:// URL instead of the local file path:

  • the URL requested is salt://dns_domain/example.net.yaml
  • the local file path (in minion cache?) contains dns:domain/example.net.yaml which is invalid on windows

Setup

  • create the per DNS domain file directory 
    mkdir /srv/salt/dns:domain
  • create a simple file in it matching the minions sal["grains.get"]("dns:domain") 
    echo -e "test-colon:\n  foo: true" > /srv/salt/dns:domain/example.net.yaml
  • create a test-colon-url.sls file to load that yaml 
    {%- load_yaml as loaded_values %}
{%-   include "dns:domain/" ~ salt["grains.get"]("dns:domain", "default") ~ ".yaml" %}
{%- endload %}

test-colon-in-salt-url:
  test.nop:
    - name: {{ loaded_values }}

Steps to Reproduce the behavior

salt 'windows-minion.example.net' state.show_sls test-colon-url
    - Rendering SLS 'base:test-colon-url' failed: Jinja error: dns:domain/example.net.yaml
      Traceback (most recent call last):
        File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
          output = template.render(**decoded_context)
        File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
          return original_render(self, *args, **kwargs)
        File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
          return self.environment.handle_exception(exc_info, True)
        File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
          reraise(exc_type, exc_value, tb)
        File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
          raise value.with_traceback(tb)
        File "<template>", line 1, in top-level template code
        File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
          rv = self._func(*arguments)
        File "<template>", line 2, in template
        File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
          raise TemplateNotFound(template)
      jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml
      
      ; line 2
      
      ---
      {%- load_yaml as loaded_values %}
      {%-   include "dns:domain/" ~ salt["grains.get"]("dns:domain", "default") ~ ".yaml" %}    <======================
      {%- endload %}
      
      test-colon-in-salt-url:
        test.nop:
          - name: {{ loaded_values }}
      [...]
      ---
ERROR: Minions returned with non-zero exit code
The debug logs on the windows minion 
2020-10-13 19:22:31,023 [salt.fileclient  :1152][DEBUG   ][4776] In saltenv 'base', looking at rel_path 'test-colon-url.sls' to resolve 'salt://test-colon-url.sls'
2020-10-13 19:22:31,023 [salt.fileclient  :1161][DEBUG   ][4776] In saltenv 'base', ** considering ** path 'c:\salt\var\cache\salt\minion\files\base\test-colon-url.sls' to resolve 'salt://test-colon-url.sls'
2020-10-13 19:22:31,023 [salt.template    :62  ][DEBUG   ][4776] compile template: c:\salt\var\cache\salt\minion\files\base\test-colon-url.sls
2020-10-13 19:22:31,023 [salt.utils.jinja :92  ][DEBUG   ][4776] Jinja search path: ['c:\\salt\\var\\cache\\salt\\minion\\files\\base']
2020-10-13 19:22:31,039 [salt.transport.zeromq:173 ][DEBUG   ][4776] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'windows-minion.example.net', 'tcp://10.1.3.5:4506', 'aes')
2020-10-13 19:22:31,039 [salt.crypt       :494 ][DEBUG   ][4776] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'windows-minion.example.net', 'tcp://10.1.3.5:4506')
2020-10-13 19:22:31,039 [salt.transport.zeromq:258 ][DEBUG   ][4776] Connecting the Minion to the Master URI (for the return server): tcp://10.1.3.5:4506
2020-10-13 19:22:31,039 [salt.transport.zeromq:1300][DEBUG   ][4776] Trying to connect to: tcp://10.1.3.5:4506
2020-10-13 19:22:31,054 [salt.utils.lazy  :102 ][DEBUG   ][4776] LazyLoaded grains.get
2020-10-13 19:22:31,054 [salt.fileclient  :1152][DEBUG   ][4776] In saltenv 'base', looking at rel_path 'dns_domain/example.net.yaml' to resolve 'salt://dns_domain/example.net.yaml'
2020-10-13 19:22:31,054 [salt.fileclient  :1161][DEBUG   ][4776] In saltenv 'base', ** considering ** path 'c:\salt\var\cache\salt\minion\files\base\dns_domain\example.net.yaml' to resolve 'salt://dns_domain/example.net.yaml'
2020-10-13 19:22:31,070 [salt.utils.templates:437 ][DEBUG   ][4776] Jinja Error
2020-10-13 19:22:31,070 [salt.utils.templates:438 ][DEBUG   ][4776] Exception:
Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml
2020-10-13 19:22:31,070 [salt.utils.templates:439 ][DEBUG   ][4776] Out: 
2020-10-13 19:22:31,070 [salt.utils.templates:440 ][DEBUG   ][4776] Line: 2
2020-10-13 19:22:31,070 [salt.utils.templates:441 ][DEBUG   ][4776] TmplStr: {%- load_yaml as loaded_values %}
{%-   include "dns:domain/" ~ salt["grains.get"]("dns:domain", "default") ~ ".yaml" %}
{%- endload %}

test-colon-in-salt-url:
  test.nop:
    - name: {{ loaded_values }}

Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml

2020-10-13 19:22:31,070 [salt.utils.templates:442 ][DEBUG   ][4776] TraceStr: Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml

2020-10-13 19:22:31,070 [salt.utils.templates:179 ][ERROR   ][4776] Rendering exception occurred
Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 166, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 445, in render_jinja_tmpl
    "Jinja error: {0}{1}".format(exc, out), line, tmplstr, trace=tracestr
salt.exceptions.SaltRenderError: Jinja error: dns:domain/example.net.yaml
Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml

; line 2

---
{%- load_yaml as loaded_values %}
{%-   include "dns:domain/" ~ salt["grains.get"]("dns:domain", "default") ~ ".yaml" %}    <======================
{%- endload %}

test-colon-in-salt-url:
  test.nop:
    - name: {{ loaded_values }}
[...]
---
2020-10-13 19:22:31,070 [salt.state       :3968][CRITICAL][4776] Rendering SLS 'base:test-colon-url' failed: Jinja error: dns:domain/example.net.yaml
Traceback (most recent call last):
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\templates.py", line 400, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "c:\salt\bin\lib\site-packages\jinja2\runtime.py", line 579, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\salt\bin\lib\site-packages\salt-3001.1-py3.7.egg\salt\utils\jinja.py", line 204, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml

; line 2

---
{%- load_yaml as loaded_values %}
{%-   include "dns:domain/" ~ salt["grains.get"]("dns:domain", "default") ~ ".yaml" %}    <======================
{%- endload %}

test-colon-in-salt-url:
  test.nop:
    - name: {{ loaded_values }}
[...]
---
2020-10-13 19:22:31,070 [salt.minion      :882 ][DEBUG   ][4776] Minion return retry timer set to 8 seconds (randomized)
2020-10-13 19:22:31,070 [salt.minion      :2154][INFO    ][4776] Returning information for job: 20201013172228836462
2020-10-13 19:22:31,070 [salt.transport.zeromq:173 ][DEBUG   ][4776] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'windows-minion.example.net', 'tcp://10.1.3.5:4506', 'aes')
2020-10-13 19:22:31,070 [salt.crypt       :494 ][DEBUG   ][4776] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'windows-minion.example.net', 'tcp://10.1.3.5:4506')
2020-10-13 19:22:31,070 [salt.transport.zeromq:258 ][DEBUG   ][4776] Connecting the Minion to the Master URI (for the return server): tcp://10.1.3.5:4506
2020-10-13 19:22:31,085 [salt.transport.zeromq:1300][DEBUG   ][4776] Trying to connect to: tcp://10.1.3.5:4506
2020-10-13 19:22:31,085 [salt.transport.zeromq:285 ][DEBUG   ][4776] Closing AsyncZeroMQReqChannel instance
2020-10-13 19:22:31,085 [salt.minion      :1999][DEBUG   ][4776] minion return: {'success': False, 'return': ['Rendering SLS \'base:test-colon-url\' failed: Jinja error: dns:domain/example.net.yaml\nTraceback (most recent call last):\n  File "c:\\salt\\bin\\lib\\site-packages\\salt-3001.1-py3.7.egg\\salt\\utils\\templates.py", line 400, in render_jinja_tmpl\n    output = template.render(**decoded_context)\n  File "c:\\salt\\bin\\lib\\site-packages\\jinja2\\asyncsupport.py", line 76, in render\n    return original_render(self, *args, **kwargs)\n  File "c:\\salt\\bin\\lib\\site-packages\\jinja2\\environment.py", line 1008, in render\n    return self.environment.handle_exception(exc_info, True)\n  File "c:\\salt\\bin\\lib\\site-packages\\jinja2\\environment.py", line 780, in handle_exception\n    reraise(exc_type, exc_value, tb)\n  File "c:\\salt\\bin\\lib\\site-packages\\jinja2\\_compat.py", line 37, in reraise\n    raise value.with_traceback(tb)\n  File "<template>", line 1, in top-level template code\n  File "c:\\salt\\bin\\lib\\site-packages\\jinja2\\runtime.py", line 579, in _invoke\n    rv = self._func(*arguments)\n  File "<template>", line 2, in template\n  File "c:\\salt\\bin\\lib\\site-packages\\salt-3001.1-py3.7.egg\\salt\\utils\\jinja.py", line 204, in get_source\n    raise TemplateNotFound(template)\njinja2.exceptions.TemplateNotFound: dns:domain/example.net.yaml\n\n; line 2\n\n---\n{%- load_yaml as loaded_values %}\n{%-   include "dns:domain/" ~ salt["grains.get"]("dns:domain", "default") ~ ".yaml" %}    <======================\n{%- endload %}\n\ntest-colon-in-salt-url:\n  test.nop:\n    - name: {{ loaded_values }}\n[...]\n---'], 'retcode': 1, 'jid': '20201013172228836462', 'fun': 'state.show_sls', 'fun_args': ['test-colon-url']}
2020-10-13 19:22:31,382 [salt.utils.process:1054][DEBUG   ][4520] Subprocess ProcessPayload-Job-20201013172228836462 cleaned up
2020-10-13 19:22:34,617 [salt.minion      :1650][INFO    ][4520] User root Executing command file.read with jid 20201013172235593957
2020-10-13 19:22:34,617 [salt.minion      :1654][DEBUG   ][4520] Command details {'fun': 'file.read', 'arg': ['/salt/var/log/salt/minion'], 'tgt': 'windows-minion.example.net'', 'jid': '20201013172235593957', 'ret': '', 'tgt_type': 'glob', 'user': 'root'}
2020-10-13 19:22:35,867 [salt.utils.process:1045][DEBUG   ][4520] Subprocess ProcessPayload-Job-20201013172235593957 added
2020-10-13 19:22:35,867 [salt.log.setup   :611 ][DEBUG   ][2592] Multiprocessing queue logging configured for the process running under PID: 2592 at log level 10
2020-10-13 19:22:35,882 [salt.loader      :784 ][DEBUG   ][2592] Grains refresh requested. Refreshing grains.
2020-10-13 19:22:35,882 [salt.config      :1899][DEBUG   ][2592] Reading configuration from c:\salt\conf\minion
2020-10-13 19:22:35,882 [salt.config      :2050][DEBUG   ][2592] Including configuration from 'c:\salt\conf\minion.d\_schedule.conf'
2020-10-13 19:22:35,882 [salt.config      :1899][DEBUG   ][2592] Reading configuration from c:\salt\conf\minion.d\_schedule.conf
2020-10-13 19:22:35,882 [salt.config      :2050][DEBUG   ][2592] Including configuration from 'c:\salt\conf\minion.d\log-level.conf'
2020-10-13 19:22:35,882 [salt.config      :1899][DEBUG   ][2592] Reading configuration from c:\salt\conf\minion.d\log-level.conf
2020-10-13 19:22:35,882 [salt.config      :2050][DEBUG   ][2592] Including configuration from 'c:\salt\conf\minion.d\startup.conf'
2020-10-13 19:22:35,882 [salt.config      :1899][DEBUG   ][2592] Reading configuration from c:\salt\conf\minion.d\startup.conf
2020-10-13 19:22:36,164 [salt.loaded.int.grains.core:3023][DEBUG   ][2592] Failed to read /proc/cmdline: [Errno 2] No such file or directory: '/proc/cmdline'
2020-10-13 19:22:36,773 [salt.loaded.int.grains.core:1403][DEBUG   ][2592] Motherboard info not available on this system
2020-10-13 19:22:36,914 [salt.loaded.int.grains.extra:61  ][DEBUG   ][2592] Loading static grains from c:\salt\conf\grains
2020-10-13 19:22:36,929 [salt.minion      :1257][INFO    ][2592] Creating minion process manager
2020-10-13 19:22:37,070 [salt.utils.lazy  :102 ][DEBUG   ][2592] LazyLoaded jinja.render
2020-10-13 19:22:37,070 [salt.utils.lazy  :102 ][DEBUG   ][2592] LazyLoaded yaml.render
2020-10-13 19:22:37,070 [salt.minion      :1791][INFO    ][2592] Starting a new job 20201013172235593957 with PID 2592
2020-10-13 19:22:37,086 [salt.utils.lazy  :106 ][DEBUG   ][2592] Could not LazyLoad direct_call.allow_missing_func: 'direct_call.allow_missing_func' is not available.
2020-10-13 19:22:37,117 [salt.utils.lazy  :102 ][DEBUG   ][2592] LazyLoaded file.read

Expected behavior

  1. the windows minion should not mangle the salt:// URL when getting files
  2. the windows minion should encode destination file path for unsupported characters

Screenshots

Versions Report

salt --versions-report on the master 
salt --versions-report
Salt Version:
           Salt: 3001.1
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: unknown
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: 1.4.4
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: 3.6.1
         pygit2: Not Installed
         Python: 3.8.5 (default, Jul 28 2020, 12:59:40)
   python-gnupg: 0.4.5
         PyYAML: 5.3.1
          PyZMQ: 18.1.1
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.2
 
System Versions:
           dist: ubuntu 20.04 focal
         locale: utf-8
        machine: x86_64
        release: 5.4.0-49-generic
         system: Linux
        version: Ubuntu 20.04 focal
salt --versions-report on the minion 
Salt Version:
           Salt: 3001.1
 
Dependency Versions:
           cffi: 1.12.2
       cherrypy: 17.4.1
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 1.0.0
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: Not Installed
   pycryptodome: 3.9.7
         pygit2: Not Installed
         Python: 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
   python-gnupg: 0.4.4
         PyYAML: 5.3.1
          PyZMQ: 18.0.1
          smmap: 2.0.5
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1
 
System Versions:
           dist:   
         locale: cp1252
        machine: AMD64
        release: 10
         system: Windows
        version: 10 10.0.18362 SP0

Additional context

@dafyddj raised concerns about windows platform in the formula community when implementing a configurable loading of YAML parameters files in map.jinja.

It's working fine on GNU/Linux minion 
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] In saltenv 'base', looking at rel_path 'test-colon-url.sls' to resolve 'salt://test-colon-url.sls'
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/test-colon-url.sls' to resolve 'salt://test-colon-url.sls'
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] compile template: /var/cache/salt/minion/files/base/test-colon-url.sls
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'pc-profs-linux.dompedago.etb1.lan', 'tcp://10.1.3.5:4506', 'aes')
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'pc-profs-linux.dompedago.etb1.lan', 'tcp://10.1.3.5:4506')
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://10.1.3.5:4506
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Trying to connect to: tcp://10.1.3.5:4506
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] LazyLoaded grains.get
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] In saltenv 'base', looking at rel_path 'dns:domain/example.net.yaml' to resolve 'salt://dns:domain/example.net.yaml'
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/dns:domain/example.net.yaml' to resolve 'salt://dns:domain/example.net.yaml'
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/test-colon-url.sls' using 'jinja' renderer: 0.02530074119567871
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Rendered data from file: /var/cache/salt/minion/files/base/test-colon-url.sls:
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: test-colon-in-salt-url:
oct. 13 19:18:51 pcubuntumate salt-minion[11772]:   test.nop:
oct. 13 19:18:51 pcubuntumate salt-minion[11772]:     - name: {'test-colon': {'foo': True}}
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: [DEBUG   ] Results of YAML rendering:
oct. 13 19:18:51 pcubuntumate salt-minion[11772]: OrderedDict([('test-colon-in-salt-url', OrderedDict([('test.nop', [OrderedDict([('name', OrderedDict([('test-colon', OrderedDict([('foo', True)]))]))])])]))])

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the salt.fileclient path handling and the salt.utils.jinja include resolution shown in the logs, then reproduce the dns:domain example on a Windows minion. Trace how the salt:// URL and the cached local path are transformed; done means the include resolves successfully without invalid Windows path characters.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.