saltstack / saltstack/salt

[BUG] Pillars are empty when running through custom execution module with salt-ssh

Open
#61,637 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Pillar Salt-SSH severity-high
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
I'm trying to access pillars inside a custom execution module using salt-ssh, but the __pillars__ dict is empty.

Using __salt__['pillar.get']() though.

Setup
I'm executing via a docker container in my local machine, targetting a Digital Ocean droplet.

# Dockerfile
ARG SALT_VERSION
FROM saltstack/salt:${SALT_VERSION}
ARG SALT_VERSION
RUN apk add --no-cache openssh-client
RUN USE_STATIC_REQUIREMENTS=1 pip3 install --no-cache-dir salt-ssh=="${SALT_VERSION}" pyopenssl
CMD ["salt-ssh"]
# composer.yaml
services:
  ssh:
    build:
      args:
        SALT_VERSION: 3004
    hostname: salt
    volumes:
      - ./etc/master:/etc/salt/master:ro
      - ./etc/roster:/etc/salt/roster:ro
      - ./roots:/srv/salt:ro
      - ./Saltfile:/root/.salt/Saltfile:ro
      - $HOME/.ssh/id_rsa:/etc/salt/pki/master/ssh/salt-ssh.rsa:ro
# /etc/salt/master

file_client: local

file_roots:
  base:
    - /srv/salt/base/salt
    - /srv/salt/formulas/mysql-formula
    - /srv/salt/formulas/users-formula
    - /srv/salt/formulas/redis-formula
    - /srv/salt/formulas/docker-formula

pillar_roots:
  base:
    - /srv/salt/base/pillar
# /etc/salt/roster
minion:
    host: my.example.net
    set_path: '/usr/local/opt/pyenv/versions/3.7.12/bin:$PATH'  # to make sure I have the same version on both sides

Here's /srv/salt/ structure:

.
├── base
│   ├── pillar
│   │   ├── ...
│   │   └── top.sls
│   └── salt
│       ├── _modules
│       │   └── helpers.py
│       ├── ...
│       └── top.sls
└── formulas
    ├── docker-formula
    ├── mysql-formula
    ├── redis-formula
    └── users-formula
# pillar/init.sls

user: myuser
# salt/_modules/helpers.py

def pillar_get(key, default=None, **options):
    return __salt__['pillar.get'](key.replace('/', ':'), default)

# to debug
def pillars():
    return ', '.join(__pillar__.keys())

Steps to Reproduce the behavior

$ docker compose run --rm ssh salt-ssh minion pillar.get user
minion:
    myuser
$ docker compose run --rm ssh salt-ssh minion helpers.pillar_get user
minion:
    None
$ docker compose run --rm ssh salt-ssh minion helpers.pillars
minion:

Expected behavior
I expect that __pillar__ inside _modules/helper.py is filled, and that __salt__['pillar.get']() works.

As you can see, it works when executing salt-ssh minion pillar.get directly. Also, salt-ssh minion pillar.items returns my pillars, so it's being matched correctly in pillar/top.sls.

Versions Report

salt-ssh --versions-report
Salt Version:
          Salt: 3004
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: 1.1.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.17
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: 1.6.1
        Python: 3.7.12 (default, Sep  8 2021, 01:55:52)
  python-gnupg: 0.4.4
        PyYAML: 5.4.1
         PyZMQ: 18.0.1
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.1
 
System Versions:
          dist: alpine 3.14.2 
        locale: UTF-8
       machine: x86_64
       release: 5.10.76-linuxkit
        system: Linux
       version: Alpine Linux 3.14.2
salt-ssh minion test.versions_report
minion:
    Salt Version:
              Salt: 3004
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: Not Installed
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.11.3
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.2
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: Not Installed
      pycryptodome: Not Installed
            pygit2: Not Installed
            Python: 3.7.12 (default, Feb 10 2022, 23:34:36)
      python-gnupg: Not Installed
            PyYAML: 5.4.1
             PyZMQ: Not Installed
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: Not Installed
     
    System Versions:
              dist: ubuntu 16.04 xenial
            locale: UTF-8
           machine: x86_64
           release: 4.4.0-148-generic
            system: Linux
           version: Ubuntu 16.04 xenial

Additional context
I've been using salt-master for a number of years, but I'm trying now to depend only on SSH. So my configs have been working for a long time, I'm just trying to make it all work with salt-ssh.

I've shutdown my master and the minion service to depend only on SSH.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with the salt-ssh execution of custom modules and compare it with direct pillar.get; inspect salt/_modules/helpers.py and the supplied master and pillar configuration. Reproduce the three commands, then verify that pillar is populated and helpers.pillar_get returns the pillar value.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.