hooks_intelligence_stats: `sona` block reports a per-process counter that resets on every MCP server restart, while the real learning signal (`hooks_post-task`) feeds a separate, persisted counter — users read "SONA 0" next to "30,622 trajectories" and conclude SONA is dead
- Dominant language
- TypeScript
- Stars
- 72.7k
- Forks
- 8.6k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 83
Description
# hooks_intelligence_stats: `sona` block reports a per-process counter that resets on every MCP server restart, while the real learning signal (`hooks_post-task`) feeds a separate, persisted counter — users read "SONA 0" next to "30,622 trajectories" and conclude SONA is dead
**Filing note:** two related findings are folded into this issue rather than split out, because both are the same root shape — a trajectory object that behaves as process-local when a caller reasonably expects it to be project-scoped: (1) the `activeTrajectories` map behind `trajectory-start`/`trajectory-end` is per-process, so a trajectory id cannot be closed out from a different MCP client/session than the one that opened it; (2) `trajectory-end`'s own `globalStatsTrajectoriesDelta` response field has been observed reporting `0` for a call that measurably moved the counter it names. If the maintainer prefers, (1) and (2) can be split from the primary per-process-counter finding below — none of the three depend on each other to be true.
**Environment**
- ruflo 3.38.19
- node v24.6.0
- macOS 26.6.2
- agentdb 3.0.0-alpha.20
**Steps to reproduce**
1. Run a project for weeks with normal `hooks_post-task` calls at task close (the standard learning-loop path).
2. Call `hooks_intelligence_stats` right after an MCP server (re)start, before any `trajectory-start`/`trajectory-end` calls have happened in that process.
3. Compare the `sona` block's counters to the `global` block's counters in the same response.
**Observed**
`sona.trajectoriesTotal` reads `0` (or a small number reflecting only calls made in the current process) immediately after restart, while `global.trajectoriesRecorded` (persisted `globalStats`, fed by every `hooks_post-task` call over the project's whole history) reads a large accumulated number — in this install, 30,622 at the same instant `sona.trajectoriesTotal` read 0. The tool itself detects and flags this as a drift condition (`consistency.notes`) but the field names (`sona.trajectoriesTotal` next to a global count in the hundreds/thousands) read as "SONA is broken" rather than "SONA hasn't processed a trajectory-end in this process yet."
A second, distinct trajectory-tracking gap compounds this: `activeTrajectories`, the map that holds a trajectory between `trajectory-start` and `trajectory-end`, is a bare in-process `Map` with no cross-process persistence — so a trajectory id returned by `trajectory-start` in one MCP client/session cannot be closed out from a different one. Live specimen from this install: a team lead's door started `traj-1788566620421-gnzpdu` (`trajectory-start`); a different seat's session later called `trajectory-end` with that id and got back `{persisted:false, note:"Trajectory not found", sonaUpdate:false}` — the id was never in that process's `activeTrajectories` map.
A third, separate reporting defect: the `globalStatsTrajectoriesDelta` field `trajectory-end` returns in its response does not reliably reflect the counter that actually moved. Two independently-fired specimens in this install: one call reported `globalStatsTrajectoriesDelta: 1` (`traj-1788566493636-9ayj8i`) with `stats.sona.trajectoriesTotal` moving as expected; a second, separately-fired call reported `globalStatsTrajectoriesDelta: 0` (`traj-1788567014412-k4qi5`) while `stats.sona.trajectoriesTotal` was independently observed moving `1 → 2` across the same call — i.e. the delta field said "no change" for a call that demonstrably changed the counter.
**Expected**
Either the two counters should track the same underlying activity (a `hooks_post-task` call should be reflected in `sona.trajectoriesTotal` the way it already is in `global.trajectoriesRecorded`), or the response should make unmistakably clear — not just in an optional `consistency.notes` array — that `sona.trajectoriesTotal` is scoped to "since this MCP server process started," not "ever." Separately: a trajectory id returned by `trajectory-start` should either be usable from any MCP client of the same project (the way its final persistence already survives process boundaries), or the docs/response shape should say plainly that trajectory ids are per-process and cannot be handed to another session. And `trajectory-end`'s `globalStatsTrajectoriesDelta` field should reliably equal the observed change in the counter it claims to describe, or should not be reported at all.
**Root cause**
Two independent objects feed from two independent MCP tool calls, and only one of them is normally called:
- `hooks_post-task` → `recordTrajectory()` → `LocalSonaCoordinator` + `ruvllm` coordinator + reasoning-bank patterns + **persisted** `globalStats.trajectoriesRecorded++` (survives process restarts, written to `.claude-flow/neural/stats.json`).
- `hooks_intelligence_trajectory-start` / `-step` / `-end` → on `-end`, calls `SONAOptimizer.processTrajectoryOutcome()`, which increments `trajectoriesProcessed` on an **in-memory, per-process** object with no persistence shown in the read range.
`hooks_intelligence_stats`' `sona` block reads `trajectoriesProcessed` from that second, per-process object — not from the first, persisted one — so it is only ever nonzero in a process where something explicitly called the `trajectory-start/-step/-end` triple, which is a separate, less-used door from the routine `hooks_post-task` call most of the learning loop actually makes.
Upstream at pinned sha `db4991967c45c6f72133dff0bb80b0a492960fc1`:
`v3/@claude-flow/cli/src/mcp-tools/hooks-tools.ts` (stats assembly):
```
3498: let sonaStats = {
...
3508: const realSona = sona.getStats();
3509: const totalRoutes = realSona.successfulRoutings + realSona.failedRoutings;
3510: sonaStats = {
3511: trajectoriesTotal: realSona.trajectoriesProcessed,
```
`v3/@claude-flow/cli/src/memory/sona-optimizer.ts` (the per-process counter, fed only by trajectory-end):
```
228: private trajectoriesProcessed = 0;
...
306: processTrajectoryOutcome(outcome: TrajectoryOutcome): {
...
363: this.trajectoriesProcessed++;
```
`v3/@claude-flow/cli/src/memory/intelligence.ts` (the tool's own acknowledgment of the drift, and the persisted counter it's being compared against):
```
921: const sonaTracksGlobalDelta = sonaStats.trajectoriesTotal - intel.trajectoriesRecorded;
...
923: if (sonaAvailable && Math.abs(sonaTracksGlobalDelta) > 2) {
924: notes.push(`sona.trajectoriesTotal (${sonaStats.trajectoriesTotal}) drifts from globalStats.trajectoriesRecorded (${intel.trajectoriesRecorded}) by ${sonaTracksGlobalDelta} — expected to track within ±1`);
...
943: source: 'sonaCoordinator (in-memory, resets per process)',
```
Installed dist (ruflo 3.38.19), identical logic:
`@claude-flow/cli/dist/src/mcp-tools/hooks-tools.js`:
```
3355: let sonaStats = {
3356: trajectoriesTotal: memoryStats.trajectories.total,
...
3364: if (sona) {
3365: const realSona = sona.getStats();
3366: const totalRoutes = realSona.successfulRoutings + realSona.failedRoutings;
3367: sonaStats = {
3368: trajectoriesTotal: realSona.trajectoriesProcessed,
```
`@claude-flow/cli/dist/src/memory/sona-optimizer.js`:
```
196: processTrajectoryOutcome(outcome) {
...
237: this.trajectoriesProcessed++;
```
`@claude-flow/cli/dist/src/memory/intelligence.js`:
```
712: const sonaTracksGlobalDelta = sonaStats.trajectoriesTotal - intel.trajectoriesRecorded;
713: const notes = [];
714: if (sonaAvailable && Math.abs(sonaTracksGlobalDelta) > 2) {
715: notes.push(`sona.trajectoriesTotal (${sonaStats.trajectoriesTotal}) drifts from globalStats.trajectoriesRecorded (${intel.trajectoriesRecorded}) by ${sonaTracksGlobalDelta} — expected to track within ±1`);
...
733: source: 'sonaCoordinator (in-memory, resets per process)',
```
`v3/@claude-flow/cli/src/mcp-tools/hooks-tools.ts` (pinned sha, the per-process trajectory map behind the `trajectory-start`/`-end` pairing bug):
```
473: const activeTrajectories = new Map();
...
2851: activeTrajectories.set(trajectoryId, trajectory);
...
2914: const trajectory = activeTrajectories.get(trajectoryId);
```
Installed dist, `@claude-flow/cli/dist/src/mcp-tools/hooks-tools.js`, identical logic:
```
391: const activeTrajectories = new Map();
...
2708: activeTrajectories.set(trajectoryId, trajectory);
...
2845: const trajectory = activeTrajectories.get(trajectoryId);
```
This install's own measured before/after (via `hooks_intelligence_stats` and `hooks_post-task`/`trajectory-end` called directly through the door): `sona.trajectoriesTotal` moved `0 → 1` only after a `trajectory-end` call was made explicitly; a same-window `hooks_post-task` call (the call most of the estate's own learning loop actually makes at task close) moved `global.trajectoriesRecorded` and `ruvllm.trajectories`/`hnsw.indexSize`, but did not move `sona.trajectoriesTotal` at all — confirming the two are fed by disjoint call paths, not merely disjoint in timing.
**Suggested fix — offered as options, this is a design choice, not a decided answer**
1. **Document it.** State plainly in the `hooks_intelligence_stats` output shape (and in user-facing docs) that `sona.trajectoriesTotal` is process-scoped and will read low/zero after any restart; point users at `global.trajectoriesRecorded` for the persisted, all-time count. Lowest-risk, no behavior change.
2. **Unify the feed.** Have `hooks_post-task`'s `recordTrajectory()` path also call (or be called alongside) `SONAOptimizer.processTrajectoryOutcome()`, so the routine learning-loop call updates both counters together. Higher-value, more invasive — needs a decision on whether `SONAOptimizer` should also persist across restarts, or whether "process-scoped SONA, all-time global" is an intentional two-tier design that should stay that way.
3. **Report both counters together with explicit scope labels** in every response (`sona.trajectoriesTotal` renamed or accompanied by e.g. `sona.trajectoriesTotalAllTime` sourced from `globalStats`), so neither reading can be mistaken for the other without opting into the raw field names.
4. **Persist `activeTrajectories` across processes** (e.g. via the memory bridge it already uses to persist a trajectory once it *ends*) so a `trajectory-start`/`trajectory-end` pair issued from different MCP clients of the same project can complete — or document explicitly that trajectory ids are single-process-only and must not be handed between sessions.
5. **Fix or drop `globalStatsTrajectoriesDelta`.** Report the per-process counter (`sona.trajectoriesTotal`, this-process-since-start) and the persisted global counter (`globalStats.trajectoriesRecorded`, all-time) as two separately-named fields the caller can diff themselves, rather than a single derived `...Delta` field that has been observed to read `0` on a call that measurably changed the counter it is named after.
**Related**
- #2977 (native controller class — different subsystem, same "documented absence, not a bug in the strict sense" shape worth noting for triage precedent)
- ADR-174 M3 (consolidate/distill worker — a third, separate object again: `consolidation.json` patterns, fed by neither of the two counters above)
**Evidence / estate provenance**
`task/task-1788563164755-ticrih/recon-c5-sona-hnsw` (ns `final`) — the "HOW CANON FEEDS SONA (three separate objects, three separate doors)" section and the measured `0 → 1` / `41 → 42` before/after; canon-defects list item D-D in the same row. Cross-referenced against `task/task-1788565562107-hrvg6b/receipts` item (6), which independently observed `promoted: 0` from a *different* stub controller (`memoryConsolidation`, #2977) in the same consolidate run — noted here only to avoid conflating that stub with the SONA counters, which are not stubbed, just disjoint.
Cross-process `activeTrajectories` specimen: `task/task-1788566146255-6q61bc/receipts` (ns `final`), `learning.trajectoryEnd` field — `trajectoryIdGiven: "traj-1788566620421-gnzpdu"` (started by a team lead's door process), result `persisted:false, note:'Trajectory not found', sonaUpdate:false` when ended from a different seat's session. `globalStatsTrajectoriesDelta` unreliability specimens: `traj-1788566493636-9ayj8i` delta `1` in `task/task-1788563164755-ticrih/recon-c5-sona-hnsw`; `traj-1788567014412-k4qi5` delta `0` (while `stats.sona.trajectoriesTotal` moved `1→2` in the same process) in `task/task-1788563164755-ticrih/quality`, item `1_sonaTrajectory`, flagged there as finding `T1`.
Contributor guide
Research direction
Start in v3/@claude-flow/cli/src/mcp-tools/hooks-tools.ts at the stats assembly and activeTrajectories map, then read memory/sona-optimizer.ts and memory/intelligence.ts. Reproduce the behavior through hooks_intelligence_stats, hooks_post-task, and trajectory-start/-end across server restarts or sessions. Done requires an agreed scope for the counters and trajectory lifecycle, with responses and documentation matching that scope consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- backend-api-design, observability
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100