saltstack / saltstack/salt

[BUG] State docker_container.running always re-create container

Open
#59,313 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug severity-high
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
docker_container.running always try to re-create existing container

Setup

grafana_container:
  docker_container.running:
    - name: grafana
    - image: grafana/grafana:7.3.7
    - working_dir: /usr/share/grafana
    - binds:
      - grafana:/var/lib/grafana
    - environment:
      ...
    - labels:
      ...
    - mem_limit: 1024M
    - replace: False
    - force: False
    - ports: 3000/tcp
    - detach: True
    - restart_policy: on-failure:3
    - require:
      - docker_volume: grafana_volume

Steps to Reproduce the behavior
First run:

/srv/salt/states # salt 'mon01*' state.apply containers.grafana
mon01:
  Name: grafana - Function: docker_volume.present - Result: Clean Started: - 14:39:20.852342 Duration: 4.757 ms
----------
          ID: grafana_container
    Function: docker_container.running
        Name: grafana
      Result: True
     Comment: Created container 'grafana'
     Started: 14:39:20.858533
    Duration: 7495.766 ms
     Changes:
              ----------
              container_id:
                  ----------
                  added:
                      77bf7c76cc97caa9b90705f11e364d757b650c434765c3e94299a7e11be55fc8
              image:
                  ----------
                  Time_Elapsed:
                      5.945162534713745
                  retcode:
                      0
              state:
                  ----------
                  new:
                      running
                  old:
                      None

Summary for mon01
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:   7.501 s

Second run:

/srv/salt/states # salt 'mon01*' state.apply containers.grafana test=True
mon01:
  Name: grafana - Function: docker_volume.present - Result: Clean Started: - 14:56:55.501558 Duration: 3.958 ms
----------
          ID: grafana_container
    Function: docker_container.running
        Name: grafana
      Result: None
     Comment: Container 'grafana' would be replaced
     Started: 14:56:55.506768
    Duration: 11.527 ms
     Changes:
              ----------
              image:
                  ----------
                  new:
                      sha256:13afb861111cf6c5b6af6fcd4917000f9a63c84ac925df666b00998ef04c35b6
                  old:
                      docker.io/grafana/grafana:7.3.7

Summary for mon01
------------
Succeeded: 2 (unchanged=1, changed=1)
Failed:    0
------------
Total states run:     2
Total run time:  15.485 ms

Expected behavior
docker_container.running should not try to re-create container there are no differences between running container and container definition in recipe.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3002.2

Dependency Versions:
          cffi: 1.11.5
      cherrypy: Not Installed
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: Not Installed
      M2Crypto: 0.35.2
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.14
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: Not Installed
        Python: 3.6.8 (default, Dec 22 2020, 19:04:08)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 19.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.3

System Versions:
          dist: centos 8
        locale: UTF-8
       machine: x86_64
       release: 4.18.0-259.el8.x86_64
        system: Linux
       version: CentOS Stream 8

Additional context
Problematic part is that docker_container.running compare 2 different values

current_image_id and image_id

https://github.com/saltstack/salt/blob/511e49fc1020f4c7230d98c98d0b91868e68fd28/salt/states/docker_container.py#L1728

where current_image_id is:

current_image_id = __salt__["docker.inspect_container"](name)["Image"]
mon01# salt-call docker.inspect_container grafana
local:
  ...
  Image:
        docker.io/grafana/grafana:7.3.7
  ...

current_image_id = "docker.io/grafana/grafana:7.3.7"

and image_id is:

image_id = _resolve_image(ret, image, client_timeout)

def _resolve_image(ret, image, client_timeout):
    """
    Resolve the image ID and pull the image if necessary
    """
    image_id = __salt__["docker.resolve_image_id"](image)
mon01# salt-call docker.resolve_image_id docker.io/grafana/grafana:7.3.7
local:
    sha256:13afb861111cf6c5b6af6fcd4917000f9a63c84ac925df666b00998ef04c35b6

image_id = "sha256:13afb861111cf6c5b6af6fcd4917000f9a63c84ac925df666b00998ef04c35b6"

sha256:13afb861111cf6c5b6af6fcd4917000f9a63c84ac925df666b00998ef04c35b6 != docker.io/grafana/grafana:7.3.7

I think there is a missing extra function docker.inspect_image after retrieving image name from running container.

So it should be:

current_image_name = __salt__["docker.inspect_container"](name)["Image"]
current_image_id = __salt__["docker.inspect_image"](current_image_name)["Id"]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read salt/states/docker_container.py around line 1728, focusing on the comparison between docker.inspect_container and docker.resolve_image_id. Reproduce the first and second state runs shown in the issue, then verify that an unchanged image no longer triggers replacement and that test mode reports no changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.