saltstack / saltstack/salt

In the state module allow pillar argument to accept newlines as whitespace

Open
#57,585 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Pillar severity-medium State-Module
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
Many functions in the state module have pillar argument. The documentation about syntax isn't very clear. It usually just says:

Custom Pillar values, passed as a dictionary of key-value pairs

From that I cannot figure out if newlines are acceptable as whitespace characters.

Setup
Have one minion and one state file (contents doesn't matter).

Steps to Reproduce the behavior
Using pillar with only spaces works fine:

# salt --state-verbose False test-dbsnap-02.xdc state.apply serverinit pillar='{"foo": "bar"}'               
test-dbsnap-02.xdc:

Summary for test-dbsnap-02.xdc
------------
Succeeded: 7
Failed:    0
------------
Total states run:     7
Total run time: 114.662 ms

However, if I use a newline instead of a space, I get an error:

# salt --state-verbose False test-dbsnap-02.xdc state.apply serverinit pillar='{"foo":        
 "bar"}'
test-dbsnap-02.xdc:
    Passed invalid arguments to state.apply: apply_() takes from 0 to 1 positional arguments but 2 were given
...

That was with the newline pressed in the command line. I get the same behavior if I use shell variable:

# ptest=$'{"foo":\n"bar"}'                                    [ERR=1]
# printf '%s\n' "$ptest"
{"foo":
"bar"}
# salt --state-verbose False test-dbsnap-02.xdc state.apply serverinit "pillar=$ptest"
test-dbsnap-02.xdc:
    Passed invalid arguments to state.apply: apply_() takes from 0 to 1 positional arguments but 2 were given
...

Note that this isn't a problem with shell parsing command line. The whole pillar=$ptest incantation was placed into quotes, so the shell would pass that as a single parameter (which includes a newline character) to the salt command.

Expected behavior
Newline should be acceptable as a whitespace character.

This would be of use when supplying contents of a JSON file (which normally contains newlines). Something like this:

# pval="$(cat file.json)"
# salt minion state.apply somestate "pillar=$pval"

Screenshots
If applicable, add screenshots to help explain your problem.

Versions Report

salt --versions-report
Salt Version:
           Salt: 3000.3
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.11.1
        libgit2: Not Installed
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.8 (default, Apr  2 2020, 13:34:55)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 15.3.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7.8.2003 Core
         locale: ISO-8859-2
        machine: x86_64
        release: 4.4.169-1.el7.elrepo.x86_64
         system: Linux
        version: CentOS Linux 7.8.2003 Core

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the state.apply command with a pillar value containing a newline, then trace how the pillar argument is parsed before reaching state.apply. The work is done when newline whitespace is accepted like spaces and a regression test covers the multiline JSON case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.