[Feature]: Isolate salt daemon writable state under /var/lib/salt/<daemon>/, keep the onedir tree read-only

Motivation

/opt/saltstack/salt/ currently serves as both the salt user's home (SALT_HOME default) and the parent of the relenv extras dir (SALT_EXTRAS_DIR default /opt/saltstack/salt/extras-<py>/). Because the daemon processes need write access to those, packaging postinst scriptlets chown -R $SALT_USER /opt/saltstack/salt. Splitting the writable state out of the code tree aligns with the existing /var/cache/salt/<daemon>/, /etc/salt/pki/<daemon>/, /var/run/salt/<daemon>/ layout and gives per-daemon isolation on multi-role hosts.

Change

New packaging flag SALT_ONEDIR_HARDEN (env var + /etc/default/salt-setup / /etc/sysconfig/salt-<daemon>-setup).

When SALT_ONEDIR_HARDEN=1, for each of minion / master / syndic / api / cloud:

• SALT_HOME default → /var/lib/salt/<daemon>/home
• SALT_EXTRAS_DIR default → /var/lib/salt/<daemon>/extras-<py>
• Postinst chowns only the two paths above; /opt/saltstack/salt/ stays root:root 0755.

Unset / 0 preserves current behavior.

salt/scripts.py::salt_pip and pkg/common/onedir/_salt_onedir_extras.py also need to honor SALT_EXTRAS_DIR at runtime — today only the packaging layer does.

Upgrade migration

With SALT_ONEDIR_HARDEN=1, postinst / %posttrans moves an existing populated /opt/saltstack/salt/extras-<py>/ into /var/lib/salt/<daemon>/extras-<py>/. No-op otherwise.

Scoping

• 3006.x / 3007.x / 3008.x: add the opt-in, default unset. Release-notes announce the flip in 3009.0.
• master (3009.0): SALT_ONEDIR_HARDEN=1 becomes the default. =0 escape hatch stays one release, then deprecates.

Same rollout as #69443, #69597.

Testing

Extend tests/pytests/pkg/ to cover, per daemon (minion/master/syndic/api/cloud) on both LTS and master defaults:

• Fresh install, SALT_ONEDIR_HARDEN unset — verify legacy chown of /opt/saltstack/salt/ on LTS; hardened layout (only per-daemon /var/lib/salt/<daemon>/{home,extras-<py>} chowned, /opt/saltstack/salt/ stays root:root 0755) on master.
• Fresh install, SALT_ONEDIR_HARDEN=1 on LTS — hardened layout.
• Fresh install, SALT_ONEDIR_HARDEN=0 on master — legacy layout; deprecation warning surfaces.
• Upgrade from legacy layout to hardened — extras contents migrate to /var/lib/salt/<daemon>/extras-<py>/, /opt/saltstack/salt/ reverts to root:root, previously salt-pip-installed packages still import at daemon runtime.
• Upgrade idempotency — re-running postinst with already-migrated layout is a no-op.
• Explicit SALT_HOME / SALT_EXTRAS_DIR overrides win over the SALT_ONEDIR_HARDEN default in both directions.
• Multi-role host (minion + master installed together) — per-daemon paths isolated; a write to one daemon's extras doesn't appear in the other's.
• Runtime salt-pip install <pkg> writes to $SALT_EXTRAS_DIR regardless of packaging default, and the daemon's Python resolves imports from that location.
• /opt/saltstack/salt/ file mode assertion in the hardened case — spot-check a sampling of files under the tree stay owned by root.

Coverage must land alongside the packaging change, not after — the default flips on 3009.0, and the LTS branches will carry the opt-in for the deprecation window, so tests exercise both branches of the gate on both branch families.

Related

#69402 / 962dd41e579 — added SALT_HOME / SALT_EXTRAS_DIR overrides this builds on.

Versions

3006.x, 3007.x, 3008.x, master.

Start with salt/scripts.py and pkg/common/onedir/_salt_onedir_extras.py to trace how SALT_EXTRAS_DIR is resolved, then inspect the packaging postinst/%posttrans logic and tests/pytests/pkg/. Run the existing package tests before extending them across daemons, branch defaults, migrations, overrides, and runtime imports. Done means hardened installs isolate writable paths, preserve legacy and override behavior, migrate idempotently, and keep the onedir tree root-owned.