saltstack / saltstack/salt

[BUG] pip.installed docker causes swarm.swarm_init to throw errors

Open
#56,006 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Execution-Module severity-medium
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description of Issue

When using pip.installed to install docker python sdk for use with the swarm module, calling the swarm.swarm_init (or any other swam function) in a module.run during this execution will fail. The states will both succeed if run in separate runs but not if they're in the same highstate (whether from the same sls file or different sls files).

Setup
pip_install_docker:
  pip.installed:
    - name: docker
start_swarm:
  module.run:
    - swarm.swarm_init:
      - advertise_addr: {{ salt['grains.get']('ipv4')[0] }}
      - listen_addr: '0.0.0.0'
      - force_new_cluster: False
Steps to Reproduce Issue

Running the above states (either as one sls or two) in the same run:

----------
          ID: pip_install_docker
    Function: pip.installed
        Name: docker
      Result: True
     Comment: Python package docker was already installed
              All specified packages are already installed
     Started: 11:02:24.950155
    Duration: 1494.427 ms
     Changes:
----------
          ID: start_swarm
    Function: module.run
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1951, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/utils/decorators/__init__.py", line 631, in _decorate
                  return self._call_function(kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/utils/decorators/__init__.py", line 334, in _call_function
                  raise error
              HTTPError: 503 Server Error: Service Unavailable for url: http+docker://localhost/v1.35/swarm
     Started: 11:02:26.448862
    Duration: 6.435 ms
     Changes:

To view the expected behavior, apply a sls file containing only the first state

          ID: install_docker_python_sdk
    Function: pip.installed
        Name: docker
      Result: True
     Comment: Python package docker was already installed
              All specified packages are already installed
     Started: 11:09:08.521908
    Duration: 1488.641 ms
     Changes:

And then another containing just the second:

                   ID: initialize_new_swarm
    Function: module.run
      Result: True
     Comment: swarm.swarm_init: Success
     Started: 11:10:49.592136
    Duration: 668.464 ms
     Changes:
              ----------
              swarm.swarm_init:
                  ----------
                  Comment:
                      Docker swarm has been initialized on  and the worker/manager Join token is below
                  Tokens:

Versions Report

Master:

Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
      dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.11
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: 0.26.8
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.17
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.16 (default, Apr  6 2019, 01:42:57)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.1.2
           RAET: Not Installed
          smmap: 2.0.5
        timelib: Not Installed
        Tornado: 5.1.1
            ZMQ: 4.3.1

System Versions:
           dist: debian 10.0
         locale: UTF-8
        machine: x86_64
        release: 4.19.0-5-amd64
         system: Linux
        version: debian 10.0

Minion:

Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed

     dateutil: 2.5.3
      docker-py: 4.1.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: 1.3.7
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1

System Versions:
           dist: debian 9.9
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-9-amd64
         system: Linux
        version: debian 9.9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read salt/state.py, salt/loader.py, and salt/utils/decorators/init.py around the traced calls; compare the same-run and separate-run behavior of pip.installed and module.run swarm.swarm_init. Done means the combined highstate initializes the Docker swarm successfully without the HTTP 503 error, with regression coverage in the relevant Salt tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.