3008.x: warn_until() allocates ~15,000 SaltStackVersion objects/sec under stress
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
Description
salt.utils.versions.warn_until() constructs a fresh salt.version.SaltStackVersion on every call for both the target-version comparison and the running-version comparison:
salt/utils/versions.py:171_version_ = salt.version.SaltStackVersion(*_version_info_)salt/utils/versions.py:143-156— target-version resolution constructs anotherSaltStackVersionper call
SaltStackVersion.__init__ builds a packaging.version.Version internally.
Impact
A memray capture on the master's EventPublisher process during 4h stress showed ~1.4 M packaging.version.Version allocations totalling 2.5 GB in a 90 s window. Every call site inside the transport hot path (e.g. TCPPubClient.__init__, TCPReqServer.__init__, MessageClient.__init__ — all deprecated aliases still instantiated during connection setup and dispatch) fires warn_until and therefore allocates 2 SaltStackVersion objects per call — even though the actual warnings.warn(...) message is filtered when PYTHONWARNINGS isn't set.
At ~15,000 calls/sec sustained, the transient allocation churn is ~28 MB/sec. Glibc arena high-water-mark grows to accommodate the peak burst, then never releases — the process's RSS is pinned above its actual working set indefinitely.
Reproduction
import tracemalloc, warnings
warnings.filterwarnings("ignore")
import salt.version, salt.utils.versions
_orig = salt.version.SaltStackVersion.__init__
n = [0]
def counting(self, *a, **kw):
n[0] += 1
return _orig(self, *a, **kw)
salt.version.SaltStackVersion.__init__ = counting
n[0] = 0
for _ in range(10_000):
salt.utils.versions.warn_until(3009, "test")
print(n[0], "SaltStackVersion() calls") # -> 20000
Proposed fix
Memoize the two SaltStackVersion constructions in warn_until. Target-version resolution has 32 slots (functools.lru_cache(maxsize=32)) which is comfortably larger than the number of distinct constant-arg call sites in the codebase; running-version resolution takes the _version_info_ tuple as key (typically a single value for the process lifetime).
After the patch, the same 10,000-call loop makes 0 SaltStackVersion constructions (100% reduction on the repeated-argument path).
Measured impact on production-shaped stress
30-min WebSocket-transport master test with the patch alone (vs 4h unpatched baseline captured in the same rig):
- EventPublisher RSS peak: 271 MB → 214 MB (-57 MB / -21%)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt/utils/versions.py at the warn_until() target-version and running-version construction points, then inspect the listed transport call sites to understand the hot path. Use the provided counting reproduction to verify repeated calls no longer construct SaltStackVersion objects, and run the relevant versions tests. Done means preserving warning behavior while eliminating repeated allocations for repeated arguments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100