Proposal: measure, and likely remove, the HermitCrab analysis memo now that #493/#494 absorb its gains

Summary

Since #493 merged and with #494 pending, the analysis memo (AnalysisScope / AnalysisStateKey, added in #456) may no longer be earning its cost. This is a proposal to measure, and — if the measurement holds — to remove it. It is not a bug report and no correctness problem is claimed.

Evidence below is measured on HC-Rust (PanGloss), the Rust port of SIL.Machine.Morphology.HermitCrab. C# hc.dll remains our oracle, so nothing here is a measurement of C#. It is evidence about the shared algorithm plus a cheap falsification test for anyone with the C# harness. Follows on from the investigation in #485.

The memo's value collapsed at one commit, and it is the port of #494 + #493

We bisected the memo's own value — t_memo_off / t_memo_on, paired and interleaved in the same binary — over a 300-word Sena corpus, uncapped, single-threaded, five pairs per point:

revision	memo on	memo off	ratio	spread, 5 pairs
parent	2,987 ms	4,392 ms	1.471	1.412 – 1.549
port of #494 + #493	2,050 ms	2,052 ms	1.001	0.932 – 1.036

Ten other candidate commits were measured (memo word/byte caps, apply_mrules/apply_templates streaming, Rc-shared alternatives, push-time alternative pruning, memo-key count saturation). None reproduces the effect.

The deterministic evidence is better than the timing

A step probe across the same boundary — steps being the algorithm's own unit of work, so this does not depend on wall clock:

	memo on	memo off	steps the memo avoided
before	712,547	1,485,331	772,784
after	436,608	502,150	65,542

A 91.5% reduction in the work gap the cache exists to close. The mechanism is straightforward: PriorityUnion and the state-keyed MergeEquivalentAnalyses remove redundant analysis work rather than making it cheaper to redo. A cache over work that no longer happens has nothing to save.

What is left does not pay for the memo's own per-lookup cost

AnalysisStateKey construction clones a full Shape, two feature structures and the per-rule unapplication map on every lookup, hit or miss. C# has no interning pool here either (AnalysisScope.cs:12-15 says so explicitly), so the same per-lookup allocation profile should apply.

On a pathological template-heavy grammar (Aweti, 44 words, 200,000-step cap) the memo is now a net cost in the port:

	wall clock	peak memory	words hitting the cap
memo off	9,683 ms	43.4 MB	24
memo on	21,262 ms	362.7 MB	16

Every capped word consumes exactly 200,000 steps by definition, so the per-word figures are comparable work-for-work: 1,136 ms per capped word with the memo against 363 ms without — each step about 3x dearer.

On Sena (300 words, uncapped) the memo was worth 2.6% of wall clock and nothing at all in peak memory: 91.969558 MB against 91.969559 MB.

The cost of removal, which is not zero

The memo saves steps, and under a step cap that is what lets some words finish at all. 8 of the 44 Aweti words completed only with the memo on. Removing it loses them; they hit the cap and return a partial result.

Anyone weighing this change under a step or time budget should price that, not just the wall clock. A grammar that currently completes inside MaxStemCount/budget limits only because of the memo will stop completing. We judged that trade worth it against 2.2x wall clock and 8.4x peak memory, but that is a judgement about our workloads, not a general one.

The prediction, and how to falsify it

#493 merged 2026-09-14 — after #485 was filed, so that issue's ~6% figure predates it. #494 is still open. Our port applied both together and we could not isolate them within this bracket; a separate five-grammar attribution run points at #494's PriorityUnion as the stronger contributor, but that used a different harness and is not reproduced here.

Concretely, for anyone with the C# benchmark harness:

1. Re-run #485's Mbugwe case on current master (which now has #493). It should already show less than 6%.
2. With #494 merged, it should go to approximately zero.
3. On any grammar where a step cap is in play, count the words that complete with the memo on and off before deciding.

If (1) or (2) fails to reproduce, the sub-attribution is wrong and we would want to know.

What we have done

We removed memoization from HC-Rust. The mechanism is gone rather than defaulted off: the memo crate, the per-parse scope threading, the --memo flag and the tuning knobs. AnalysisStateKey itself survives, because MergeEquivalentAnalyses keys its fold on it — that is semantics, not caching, and it stays either way. The three tests that compared memo-on against memo-off were deleted rather than adjusted, since with one execution strategy left they can only assert a tautology.

If the C# measurements reproduce, the same removal would delete AnalysisScope.cs, its two in-flight guards and the replay path, and leave AnalysisStateKey in place for the merge. Happy to open that PR once someone confirms the numbers on the C# side — I did not want to send a removal patch ahead of a measurement on the engine it would change.

Limits of this evidence

• HC-Rust, not hc.dll. Different language, allocator and GC behaviour; the C# magnitudes could differ even if the mechanism holds.
• 300 of 6,146 Sena words; the Aweti run is 44 words of one grammar.
• #494 is not isolated from #493 in the bisect bracket.
• Peak memory was not re-measured inside the bisect sweep itself.
• Mbugwe uncapped does not terminate in our port at all (killed at 11,016 CPU-seconds), memo or no memo, so it contributed no bisect point.

Start by rerunning the #485 Mbugwe case with the C# benchmark harness on current master, then repeat after #494 if it merges. Inspect AnalysisScope.cs, its two in-flight guards, and the replay path; keep AnalysisStateKey for MergeEquivalentAnalyses. Done means confirming the measurements and removing the C# memoization path only if they reproduce.