perf_check_for_memory_leak reports nonsensical "days" value in memleak message
- Dominant language
- Python
- Stars
- 174
- Forks
- 225
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 14
Description
The memory-leak check in CIME/SystemTests/system_tests_common.py reports the elapsed time in "days" using arithmetic that is not actually days. For any run longer than a few days, the reported number is off by orders of magnitude and confusing to users.
### Example
A 5-year SMS_Ly5 ELM test produced this TestStatus.log entry:
memleak detected, memory went from 1499.740000 to 1578.320000 in 41129 days
The test only ran for 5 model years (~1825 days), not 41,129.
### Root cause
perf_get_memory_list (in CIME/baselines/performance.py) parses cpl.log lines of the form:
`... model date = 00051231 ... memory = 1578.320 MB ... highwater ...`
using the regex
`meminfo = re.compile(r".*model date =\s+(\w+).*memory =\s+(\d+\.?\d+).*highwater")`
and stores the first capture group as float(m.group(1)). That value is the model date encoded as YYYYMMDD (e.g. 51231.0), not a day count.
Then in perf_check_for_memory_leak (CIME/SystemTests/system_tests_common.py, around line 1109):
```
# last day - second day, skip first day, can be too low while initializing
elapsed_days = int(memlist[-1][0]) - int(memlist[1][0])
```
This subtracts the two encoded stamps directly. For the run above, the second sample was model date = 00010102 and the last sample was model date = 00051231, so:
51231 - 10102 = 41129
which is exactly the "41129 days" printed in the message. The result mixes years (×10000), months (×100), and day-of-month, so it grows by ~10000 per model year plus ~100 per model month and is not a meaningful count of days.
### Impact
- Cosmetic / user-facing only. The pass/fail decision itself is unaffected — that uses finalmem/originalmem and the tolerance ratio, both of which are correct.
- The MEMLEAK TestStatus.log message is misleading, and can send developers looking for a bug in the wrong place ("why does it think my 5-year run went for 100+ years?").
- Any downstream tooling that parses this comment and interprets the number as days will be wrong for any run longer than a few days.
### Suggested fix
Decode the YYYYMMDD stamp back into (year, month, day) before differencing, then report the elapsed time in years/months/days (dropping zero components), e.g.:
memleak detected, memory went from 1499.740000 to 1578.320000 in 4 years, 11 months, 29 days
The same message should be emitted by any component-specific config.perf_check_for_memory_leak override so the wording stays consistent.
Contributor guide
Research direction
Start with perf_get_memory_list in CIME/baselines/performance.py and perf_check_for_memory_leak around line 1109 in CIME/SystemTests/system_tests_common.py. Trace how the YYYYMMDD value is stored and how component-specific config.perf_check_for_memory_leak overrides emit the message. Done means the log reports a meaningful years/months/days duration consistently while the existing memory-leak pass/fail calculation remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100