saltstack / saltstack/salt

[BUG] Windows docker_container unable to intrepret API event

Open
#59,087 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Confirmed severity-medium Windows
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Migrated from https://github.com/saltstack-formulas/docker-formula/issues/262

Your setup

Formula commit hash / release tag
Versions reports (master & minion)
Salt Version:
           Salt: 3002

Dependency Versions:
           cffi: 1.12.2
       cherrypy: 17.4.1
       dateutil: 2.8.0
      docker-py: 4.4.0
          gitdb: 2.0.5
      gitpython: 2.1.10
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 1.0.0
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: Not Installed
   pycryptodome: 3.9.8
         pygit2: Not Installed
         Python: 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
   python-gnupg: 0.4.4
         PyYAML: 5.3.1
          PyZMQ: 18.0.1
          smmap: 2.0.5
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist: cygwin_nt-10.0 3.1.7
         locale: cp1252
        machine: AMD64
        release: 10
         system: Windows
        version: 10 10.0.19041 SP0
Pillar / config used
docker:
  wanted:
    - docker

  containers:
    running:
      - multi-cloud-api
      - multi-cloud-backend
      - multi-cloud-file
      - multi-cloud-block
      - multi-cloud-s3
      - multi-cloud-dataflow
      - multi-cloud-datamover

    multicloudapi:
      image: "sodafoundation/multi-cloud-api"
    multi-cloud-backend:
      image: "sodafoundation/multi-cloud-backend"
    multi-cloud-file:
      image: "sodafoundation/multi-cloud-file"
    multi-cloud-block:
      image: "sodafoundation/multi-cloud-block"
    multi-cloud-s3:
      image: "sodafoundation/multi-cloud-s3"
    multi-cloud-dataflow:
      image: "sodafoundation/multi-cloud-dataflow"
    multi-cloud-datamover:
      image: "sodafoundation/multi-cloud-datamover"

Bug details

Describe the bug

The state docker_container.running fails 50-100% of the time on Windows.

          ID: docker-containers-multi-cloud-datamover-running
    Function: docker_container.running
        Name: multi-cloud-datamover
      Result: False
     Comment: Failed to pull sodafoundation/multi-cloud-datamover: Unable to interpret API event: 'b'{"status":"Verifying Checksum","progressDetail":{},"id":"
d8217ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8217ebeaff5"}\r\n''. Additional info follows:

              Error:
                  Extra data: line 2 column 1 (char 73)
     Started: 23:43:35.290962
    Duration: 21133.161 ms
     Changes:
Steps to reproduce the bug

An API Event with one dictionary is OK:

[DEBUG   ] pull event: b'{"status":"Downloading","progressDetail":{"current":10193332,"total":10324842},"progress":"[=========================================
========\\u003e ]  10.19MB/10.32MB","id":"d8217ebeaff5"}\r\

An API Event with two dictionaries is NOK:

[DEBUG   ] pull event: b'{"status":"Verifying Checksum","progressDetail":{},"id":"d8217ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8
217ebeaff5"}\r\n'
[ERROR   ] Failed to pull sodafoundation/multi-cloud-datamover: Unable to interpret API event: 'b'{"status":"Verifying Checksum","progressDetail":{},"id":"d82
17ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8217ebeaff5"}\r\n''. Additional info follows:

Error:
    Extra data: line 2 column 1 (char 73)
Expected behavior
Attempts to fix the bug

I tried setting retry on the state but the same result.

Additional context

DEBUG LOGGING

[DEBUG   ] Attempting to run docker-py's "pull" function with args=('sodafoundation/multi-cloud-datamover',) and kwargs={'stream': True}
[DEBUG   ] Looking for auth config
[DEBUG   ] Using credentials store "desktop"
[DEBUG   ] Looking for auth entry for 'https://index.docker.io/v1/'
[DEBUG   ] Found auth config
[DEBUG   ] http://localhost:None "POST /v1.40/images/create?tag=latest&fromImage=sodafoundation%2Fmulti-cloud-datamover HTTP/1.1" 200 None
[DEBUG   ] pull event: b'{"status":"Pulling from sodafoundation/multi-cloud-datamover","id":"latest"}\r\n'
[DEBUG   ] pull event: b'{"status":"Already exists","progressDetail":{},"id":"df20fa9351a1"}\r\n'
[DEBUG   ] pull event: b'{"status":"Pulling fs layer","progressDetail":{},"id":"d8217ebeaff5"}\r\n'
[DEBUG   ] pull event: b'{"status":"Downloading","progressDetail":{"current":105704,"total":10324842},"progress":"[\\u003e
              ]  105.7kB/10.32MB","id":"d8217ebeaff5"}\r\n'
[DEBUG   ] pull event: b'{"status":"Downloading","progressDetail":{"current":212466,"total":10324842},"progress":"[=\\u003e
              ]  212.5kB/10.32MB","id":"d8217ebeaff5"}\r\n'
[DEBUG   ] pull event: b'{"status":"Downloading","progressDetail":{"current":326622,"total":10324842},"progress":"[=\\u003e
              ]  326.6kB/10.32MB","id":"d8217ebeaff5"}\r\n'
... cut ...
[DEBUG   ] pull event: b'{"status":"Downloading","progressDetail":{"current":10193332,"total":10324842},"progress":"[=========================================
========\\u003e ]  10.19MB/10.32MB","id":"d8217ebeaff5"}\r\n'
[DEBUG   ] pull event: b'{"status":"Verifying Checksum","progressDetail":{},"id":"d8217ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8
217ebeaff5"}\r\n'
[ERROR   ] Failed to pull sodafoundation/multi-cloud-datamover: Unable to interpret API event: 'b'{"status":"Verifying Checksum","progressDetail":{},"id":"d82
17ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8217ebeaff5"}\r\n''. Additional info follows:

Error:
    Extra data: line 2 column 1 (char 73)
[INFO    ] Completed state [multi-cloud-datamover] at time 23:43:56.424123 (duration_in_ms=21133.161)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the docker_container.running entry point and trace the docker-py pull stream handling described in the logs. Reproduce the Windows case with two newline-delimited API event dictionaries, then verify that the pull completes without the reported JSON “Extra data” error; no specific source file or test is named.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.