[BUG] 2019.2 Unable to get the return result from job_list when the Salt minion has lost connection to master in async mode
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Unable to get the return result from job_list when the Salt minion has lost connection to master in async mode
Steps to Reproduce Issue
I am trying to issue salt commands on a number of minions asynchronously and get the result using list_job function.
The problem is when the connection to a minion is not established, the list_job return only the result of the connected minion but there is no mention of the error from the other minion that failed to connect
The correct error is returned when the command is run synchronously:
[root@436b9c0268ea /]# salt '*' cmd.run ls
8a4c0aa0e395:
anaconda-ks.cfg
c4ec0b9e267a:
Minion did not return. [No response]
However when the same id executed with --async flag, the unreachable error is lost/not shown
[root@436b9c0268ea /]# salt '*' cmd.run ls --async
Executed command with job ID: 20190712104159308542
[root@436b9c0268ea /]# salt-run jobs.list_job 20190712104159308542
Arguments:
- ls
Function:
cmd.run
Minions:
- 8a4c0aa0e395
- c4ec0b9e267a
Result:
----------
8a4c0aa0e395:
----------
retcode:
0
return:
anaconda-ks.cfg
success:
True
StartTime:
2019, Jul 12 10:41:59.308542
Target:
*
Target-type:
glob
User:
root
jid:
20190712104159308542
Is there a way to see the unreachable result in the list_job return using asynchronous mode. It would be good to know in case of a long running job. Currently, there seems to be ambiguity whether the command is running or not.
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: 1.2.5
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.5 (default, Jun 20 2019, 20:27:34)
python-gnupg: Not Installed
PyYAML: 3.10
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.6.1810 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-957.27.2.el7.x86_64
system: Linux
version: CentOS Linux 7.6.1810 Core
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with salt '*' cmd.run ls --async, then inspect the jobs.list_job entry point and compare its output with synchronous execution when a minion is unreachable. Done means asynchronous job results clearly report the failed minion's no-response error instead of listing only connected minions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100