Macaulay2 / Macaulay2/M2

Reloading a package leaks entries into the Attributes table

Open
#4,536 0 comments 0 reactions 0 assignees View on GitHub
bugs directory Core Memory Leak
Dominant language
Macaulay2
Stars
435
Forks
297
Avg merge
4d 20h
Merged PRs (30d)
11

Description

This issue was triaged from [`bugs/dan/0-package-garbage-collection`](https://github.com/Macaulay2/M2/blob/388c1ff0ce30d83751dea7bc7eac77fdc1305dd7/bugs/dan/0-package-garbage-collection), one of the 857 files removed from the pre-GitHub `bugs/` tree by [`d2c8d27826`](https://github.com/Macaulay2/M2/commit/d2c8d27826) and catalogued in [#36](https://github.com/Macaulay2/M2/issues/36). **The commentary below was written by Claude (Claude Opus 5, via Claude Code)**, not by @d-torrance, whose account posted it -- please weigh it accordingly.

The original file, verbatim (67 lines)

```text
--*-M2-*-

reloading a package can lead to a memory leak:

set a memory limit for this test:

ulimit -v 500000 -- but this doesn't work on Mac OS X
M2

while true do value ///
newPackage ("foo",Reload=>true)
XX = new Type of BasicList
globalAssignment XX
xx = new XX from {Bag {10000000 : 0}} -- about 40 MB
<< "." << flush;
///

debug Core
peek Attributes -- shows them stashed there

another memory leak associate with reloading a package:

all global variables are stored in the single global frame:

i1 : debug Core

i2 : examine asdf777
symbol closure :
symbol : asdf777
position : stdio:2:9:(3):
frameID : 0 <=====
frameindex : 2210 <=====
lookupCount : 1
protected : false
thread : false
frames bound : 0 @4311789024 (NR) [2211 of 4095]

i3 : dictionary asdf777

o3 = User#"private dictionary"

o3 : GlobalDictionary

i4 : newPackage "Foo"

o4 = Foo

o4 : Package

i5 : debug Core

i6 : examine asd46464
symbol closure :
symbol : asd46464
position : stdio:6:9:(2):
frameID : 0 <=====
frameindex : 2214 <=====
lookupCount : 1
protected : false
thread : false
frames bound : 0 @4311789024 (NR) [2215 of 4095]

i7 : dictionary asd46464

o7 = Foo#"private dictionary"

o7 : GlobalDictionary
```

### Where it stands today

The `Attributes` half reproduces and is measurable:

| | `Attributes` entries |
| --- | ---: |
| before | 2682 |
| after 30 × `newPackage(..., Reload => true)` | 2806 |

About **four entries leaked per reload**, never reclaimed.

### Why it is worse than a slow leak

`getAttributes` scans the whole table, so a growing `Attributes` is also a growing cost on every
lookup that consults it — which is the mechanism that would eventually make `0-slowness`'s complaint
true again. A leak that also degrades a hot path deserves attention ahead of its size.

### Scope

The file describes two leaks; this is the one that reproduces cleanly and can be counted. The other —
the 40 MB-per-cycle growth under `ulimit -v` — needs a platform where the limit is enforced, which the
file itself notes does not include macOS.

### Not covered elsewhere

**#488** and **#2562** are engine and sequence allocation respectively, so neither covers this.

`open` · disposition `issue` · source of truth: [`bug-triage/catalog.tsv`](https://github.com/d-torrance/M2/blob/bug-triage/bug-triage/catalog.tsv)

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the leak with repeated newPackage("foo", Reload => true) calls and inspect the Attributes table using debug Core and getAttributes. Start from the package-reload and Attributes handling reached by that entry point; done means repeated reloads no longer increase the counted entries and lookup cost does not grow from this behavior.

Written by the indexing model from the issue text.

Assessment

Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.