All of the returners (excluding mysql and local_cache) are not handling the "req" job id correctly
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description of Issue/Question
Hiya. I'm the author of PR #51363 (etcd returner), and I notice that the mysql returner specially handles the case when the job id is set to "req". This job id seems to be submitted by a minion when it is activated independant of a command from the master. (Relevant code: https://github.com/saltstack/salt/blob/develop/salt/returners/mysql.py#L294)
The way this magic job-id is handled by the returner seems to be that when it is seen, to replace it with a new job id generated from returner.prep_jid. However, it appears that only the mysql and local_cache returners handle this job id explicitly, and everything else does not tamper with it at all. Which returner is handling it correctly?
Is this something that a returner is actually not supposed to receive and a result of a bug or misconfiguration? Most of the logic that messes with this magic job-id immediately replaces it with a call to prep_jid, however there are a few cases where this logic is missed which results in a returner receiving this 'req' job...
Setup
To view the job-id in use, simply set the startup_states in a minion to execute some state that takes a while (so that you can race against the state being applied). Configure any of the non-local returners so that you can manually check the job being created.
Steps to Reproduce Issue
As the state is being applied, when you view the active jobs salt-run jobs.active you'll see that the job id for the state that was started by the minion has the job-id of "req".
Versions Report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: 1.11.5
cherrypy: Not Installed
dateutil: 2.8.0
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.11
ioflo: 1.7.5
Jinja2: 2.10
libgit2: 0.27.8
libnacl: 1.6.1
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pycryptodome: 3.7.0
pygit2: 0.27.4
Python: 2.7.15 (default, Oct 15 2018, 15:26:09)
python-gnupg: Not Installed
PyYAML: 4.2
PyZMQ: 17.0.0
RAET: 0.6.8
smmap: 2.0.3
timelib: Not Installed
Tornado: 5.0.2
ZMQ: 4.1.6
System Versions:
dist: fedora 29 Twenty Nine
locale: ANSI_X3.4-1968
machine: x86_64
release: 4.14.96-coreos
system: Linux
version: Fedora 29 Twenty Nine
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with salt/returners/mysql.py around line 294 and compare how the other returners process the "req" job id. Reproduce the startup_states case with a long-running state, inspect salt-run jobs.active, and check the resulting job in a configured non-local returner. Done means the expected handling of "req" is established and the affected returners behave consistently or the issue is narrowed to the responsible layer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100