[BUG] Windows docker_container unable to intrepret API event
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Migrated from https://github.com/saltstack-formulas/docker-formula/issues/262
Your setup
Formula commit hash / release tag
Versions reports (master & minion)
Salt Version:
Salt: 3002
Dependency Versions:
cffi: 1.12.2
cherrypy: 17.4.1
dateutil: 2.8.0
docker-py: 4.4.0
gitdb: 2.0.5
gitpython: 2.1.10
Jinja2: 2.10.1
libgit2: Not Installed
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 1.0.0
mysql-python: Not Installed
pycparser: 2.19
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: Not Installed
Python: 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
python-gnupg: 0.4.4
PyYAML: 5.3.1
PyZMQ: 18.0.1
smmap: 2.0.5
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist: cygwin_nt-10.0 3.1.7
locale: cp1252
machine: AMD64
release: 10
system: Windows
version: 10 10.0.19041 SP0
Pillar / config used
docker:
wanted:
- docker
containers:
running:
- multi-cloud-api
- multi-cloud-backend
- multi-cloud-file
- multi-cloud-block
- multi-cloud-s3
- multi-cloud-dataflow
- multi-cloud-datamover
multicloudapi:
image: "sodafoundation/multi-cloud-api"
multi-cloud-backend:
image: "sodafoundation/multi-cloud-backend"
multi-cloud-file:
image: "sodafoundation/multi-cloud-file"
multi-cloud-block:
image: "sodafoundation/multi-cloud-block"
multi-cloud-s3:
image: "sodafoundation/multi-cloud-s3"
multi-cloud-dataflow:
image: "sodafoundation/multi-cloud-dataflow"
multi-cloud-datamover:
image: "sodafoundation/multi-cloud-datamover"
Bug details
Describe the bug
The state docker_container.running fails 50-100% of the time on Windows.
ID: docker-containers-multi-cloud-datamover-running
Function: docker_container.running
Name: multi-cloud-datamover
Result: False
Comment: Failed to pull sodafoundation/multi-cloud-datamover: Unable to interpret API event: 'b'{"status":"Verifying Checksum","progressDetail":{},"id":"
d8217ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8217ebeaff5"}\r\n''. Additional info follows:
Error:
Extra data: line 2 column 1 (char 73)
Started: 23:43:35.290962
Duration: 21133.161 ms
Changes:
Steps to reproduce the bug
An API Event with one dictionary is OK:
[DEBUG ] pull event: b'{"status":"Downloading","progressDetail":{"current":10193332,"total":10324842},"progress":"[=========================================
========\\u003e ] 10.19MB/10.32MB","id":"d8217ebeaff5"}\r\
An API Event with two dictionaries is NOK:
[DEBUG ] pull event: b'{"status":"Verifying Checksum","progressDetail":{},"id":"d8217ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8
217ebeaff5"}\r\n'
[ERROR ] Failed to pull sodafoundation/multi-cloud-datamover: Unable to interpret API event: 'b'{"status":"Verifying Checksum","progressDetail":{},"id":"d82
17ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8217ebeaff5"}\r\n''. Additional info follows:
Error:
Extra data: line 2 column 1 (char 73)
Expected behavior
Attempts to fix the bug
I tried setting retry on the state but the same result.
Additional context
DEBUG LOGGING
[DEBUG ] Attempting to run docker-py's "pull" function with args=('sodafoundation/multi-cloud-datamover',) and kwargs={'stream': True}
[DEBUG ] Looking for auth config
[DEBUG ] Using credentials store "desktop"
[DEBUG ] Looking for auth entry for 'https://index.docker.io/v1/'
[DEBUG ] Found auth config
[DEBUG ] http://localhost:None "POST /v1.40/images/create?tag=latest&fromImage=sodafoundation%2Fmulti-cloud-datamover HTTP/1.1" 200 None
[DEBUG ] pull event: b'{"status":"Pulling from sodafoundation/multi-cloud-datamover","id":"latest"}\r\n'
[DEBUG ] pull event: b'{"status":"Already exists","progressDetail":{},"id":"df20fa9351a1"}\r\n'
[DEBUG ] pull event: b'{"status":"Pulling fs layer","progressDetail":{},"id":"d8217ebeaff5"}\r\n'
[DEBUG ] pull event: b'{"status":"Downloading","progressDetail":{"current":105704,"total":10324842},"progress":"[\\u003e
] 105.7kB/10.32MB","id":"d8217ebeaff5"}\r\n'
[DEBUG ] pull event: b'{"status":"Downloading","progressDetail":{"current":212466,"total":10324842},"progress":"[=\\u003e
] 212.5kB/10.32MB","id":"d8217ebeaff5"}\r\n'
[DEBUG ] pull event: b'{"status":"Downloading","progressDetail":{"current":326622,"total":10324842},"progress":"[=\\u003e
] 326.6kB/10.32MB","id":"d8217ebeaff5"}\r\n'
... cut ...
[DEBUG ] pull event: b'{"status":"Downloading","progressDetail":{"current":10193332,"total":10324842},"progress":"[=========================================
========\\u003e ] 10.19MB/10.32MB","id":"d8217ebeaff5"}\r\n'
[DEBUG ] pull event: b'{"status":"Verifying Checksum","progressDetail":{},"id":"d8217ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8
217ebeaff5"}\r\n'
[ERROR ] Failed to pull sodafoundation/multi-cloud-datamover: Unable to interpret API event: 'b'{"status":"Verifying Checksum","progressDetail":{},"id":"d82
17ebeaff5"}\r\n{"status":"Download complete","progressDetail":{},"id":"d8217ebeaff5"}\r\n''. Additional info follows:
Error:
Extra data: line 2 column 1 (char 73)
[INFO ] Completed state [multi-cloud-datamover] at time 23:43:56.424123 (duration_in_ms=21133.161)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the docker_container.running entry point and trace the docker-py pull stream handling described in the logs. Reproduce the Windows case with two newline-delimited API event dictionaries, then verify that the pull completes without the reported JSON “Extra data” error; no specific source file or test is named.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100