3008.x: setting worker_pools_enabled=False crashes zmq_device via RequestRouter
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
On 3008.x, setting worker_pools_enabled: False in the master config to fall back to the legacy single-queue routing path (salt.transport.zeromq.RequestServer.zmq_device) causes the RequestServer.zmq_device process to crash in a tight restart loop, rendering the master unable to answer any minion requests.
Root cause
zmq_device unconditionally constructs a salt.master.RequestRouter (salt/transport/zeromq.py:602):
router = salt.master.RequestRouter(
self.opts, secrets=secrets or getattr(self, "secrets", None)
)
RequestRouter._build_routing_table (salt/master.py:1612) then validates the worker_pools config and raises:
ValueError: Worker pool configuration must have exactly one pool with catchall ('*') in its commands.
because worker_pools_enabled: False causes get_worker_pools_config to return None without leaving a valid worker_pools layout for the router constructor.
The legacy zmq_device path is the ONLY path exercised when pools are disabled; it should not depend on a valid pool configuration.
Reproduction
# /etc/salt/master
worker_pools_enabled: False
Then start the master. Log fills with:
[ERROR ][10727] An un-handled exception from the multiprocessing process 'RequestServer.zmq_device' was caught:
File ".../salt/transport/zeromq.py", line 602, in zmq_device
router = salt.master.RequestRouter(...)
File ".../salt/master.py", line 1577, in __init__
self._build_routing_table()
File ".../salt/master.py", line 1612, in _build_routing_table
raise ValueError(
ValueError: Worker pool configuration must have exactly one pool with catchall ('*') in its commands.
ProcessManager restarts the crashed zmq_device every 10s indefinitely. salt '*' test.ping times out.
Workaround
Also add an explicit worker_pools catchall alongside worker_pools_enabled: False:
worker_pools_enabled: False
worker_pools:
default:
worker_count: 3
commands: ["*"]
Suggested fix
Either
zmq_deviceskipsRequestRouterconstruction whennot self.opts.get("worker_pools_enabled", True), orRequestRouter.__init__returns a no-op router (or is not constructed) when pools are disabled.
Version: 3008.2+152.g4a10864f6a (CI RPM from PR #69908).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/transport/zeromq.py at RequestServer.zmq_device and trace its RequestRouter construction into salt/master.py, especially _build_routing_table. Reproduce with worker_pools_enabled: False, then verify the master stays running and responds to salt '*' test.ping without requiring an explicit worker_pools catchall.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100