saltstack / saltstack/salt

[Bug]: deltaproxy never tears down its sub-proxies, abandoning their channels, schedules and beacons on every stop

Open
#70,221 1 comment 0 reactions 0 assignees View on GitHub

@ggiesen is already working on this.

Since Sep 2, 2026.

Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description

A deltaproxy never tears down its sub-proxies. salt/metaproxy/deltaproxy.py builds them into self.deltaproxy_objs during post_master_init, and nothing in the shutdown path ever touches that dict again -- grep deltaproxy_objs shows it is only created, populated, and iterated for tune-in and job dispatch.

salt.minion.ProxyMinion has no destroy() of its own, so it inherits Minion.destroy(), which tears down only the instance it is called on. Each sub-proxy is a full ProxyMinion with its own req_channel (assigned in post_master_init), schedule, beacons and periodic callbacks, and all of those are abandoned on every stop or restart.

Setup

salt 3008.2, one control proxy dpcontrol with three sub-proxies (minion1, minion2, minion3) using the dummy proxytype.

Note this is only observable once the proxy actually reaches its teardown. On an unpatched 3008.2 the daemon exits before the scheduled graceful shutdown runs at all (filed separately as #70217), so apply that fix first or nothing is torn down regardless.

Steps to reproduce

Instrument Minion.destroy to record self.opts["id"], start the deltaproxy, and send it a SIGTERM.

Observed: exactly one teardown.

destroy dpcontrol

Expected: each sub-proxy is torn down as well.

destroy minion1
destroy minion2
destroy minion3
destroy dpcontrol
A related question, not part of this report

While tracing this I noticed that the proxymodule's own shutdown() is never invoked on daemon stop for any proxy, not just sub-proxies. The only callers anywhere in salt/ are salt/modules/junos.py and salt/modules/status.py. So a proxymodule that opens a device session has no teardown hook that salt itself calls when the daemon stops.

That may well be deliberate, and changing it would alter behaviour for every proxy deployment rather than just deltaproxy, so I have deliberately not touched it. Flagging it in case it is not intentional.

Versions Report
Salt Version:
           Salt: 3008.2
Python Version:
         Python: 3.10
Salt Package Information:
   Package Type: onedir

Reproduces against the current 3008.x branch head.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in salt/metaproxy/deltaproxy.py, then trace ProxyMinion and the inherited Minion.destroy() shutdown path, including self.deltaproxy_objs. Reproduce the SIGTERM scenario after applying the separate graceful-shutdown fix; done means the control proxy and every sub-proxy are torn down, with their channels, schedules, beacons, and callbacks no longer abandoned.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.