danielmiessler / danielmiessler/LifeOS

`memory-retrievals.jsonl` is graded by health but never written by anything

Open Beginner friendly
#2,081 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
19k
Forks
2.5k
Avg merge
8d 17h
Merged PRs (30d)
1

Description

## Summary

`MemoryHealthCheck` grades the memory subsystem on retrieval evidence read from
`LIFEOS/MEMORY/OBSERVABILITY/memory-retrievals.jsonl`. Two tools read that file and the Cortex
contract specifies it. **Nothing in the payload writes it.**

The result is a warning that can never clear on any install. `retrieval-missing` fires on a
healthy system, every day, forever.

Retrieval itself works correctly. Only the logging leg is missing.

- **LifeOS:** 7.40.4
- **Cortex (Memory):** v8.3.0
- **Platform:** Windows 10 Pro, bun 1.3.10

## Evidence

**Readers — 2:**

- `LIFEOS/TOOLS/CortexHealth.ts:206` — `const retrievalLog = join(obs, "memory-retrievals.jsonl"), retrieval = readJsonl(retrievalLog), retrievalLast = retrieval.rows.at(-1);`
- `LIFEOS/TOOLS/MemoryStatus.ts:44` — `const RETRIEVALS_PATH = pathJoin(OBS_DIR, "memory-retrievals.jsonl");`

**Specified in the contract:**

`LIFEOS/DOCUMENTATION/Memory/CortexContract.md:211` —

> Retrieval evidence comes from the latest valid `memory-retrievals.jsonl` row.

Also referenced in `Memory/MemorySystem.md` and `Observability/ObservabilitySystem.md`.

**Writers — 0:**

`LIFEOS/TOOLS/MemoryRetriever.ts` contains no write of any kind: no `writeFile`, no `appendFile`,
no reference to `OBSERVABILITY`. Neither does `hooks/MemoryTurnStart.hook.ts`, its only caller.
A grep for `memory-retrievals` across the install matches five files — the two readers above and
three documentation files.

## Reproduce

On any install, after normal use:

```bash
ls ~/.claude/LIFEOS/MEMORY/OBSERVABILITY/memory-retrievals.jsonl
# No such file or directory

grep -rl "memory-retrievals" ~/.claude/LIFEOS --include=*.ts
# LIFEOS/TOOLS/CortexHealth.ts
# LIFEOS/TOOLS/MemoryStatus.ts <- both read

bun ~/.claude/LIFEOS/TOOLS/MemoryHealthCheck.ts
```

```json
{
"id": "retrieval-missing",
"severity": "warn",
"message": "No retrieval evidence within 86400000ms freshness window.",
"detail": { "status": "missing", "thresholdMs": 86400000 }
}
```

Meanwhile retrieval is demonstrably working — `` blocks render on every turn from
the same code path.

## Why it matters

A permanently-on warning is worse than no warning. It trains operators to read `warn` as the
normal state, which is how the next real finding gets skipped. It also makes the retrieval leg
unobservable: there is no way to tell a retriever returning nothing from a retriever never called,
which is exactly what #1573 turned out to be.

## Proposed fix

Log one row per retrieval at the call site. `getRelevantContext` already returns everything the
readers want, and the write follows the same pattern `MemoryReviewer.ts:61` uses for
`reviewer-runs.jsonl`.

In `hooks/MemoryTurnStart.hook.ts`, around the existing call at line 121:

```diff
if (prompt.trim().length > 0) {
try {
const ground = getRelevantContext(prompt, { topK: 5, threshold: 0.20 });
+ appendFileSync(
+ pathResolve(CLAUDE_ROOT, "LIFEOS/MEMORY/OBSERVABILITY/memory-retrievals.jsonl"),
+ JSON.stringify({
+ ts: new Date().toISOString(),
+ results: ground.results.length,
+ totalSearched: ground.totalSearched,
+ cached: ground.cached,
+ injected: ground.markdownBlock.length > 0,
+ }) + "\n",
+ );
if (ground.markdownBlock) {
process.stdout.write(`\n${ground.markdownBlock}\n\n`);
}
} catch (e) {
process.stderr.write(`MemoryTurnStart ground error: ${(e as Error)?.message || String(e)}\n`);
}
}
```

Deliberately no query text and no result content in the row: this file is diagnostic, and the
prompt is the most sensitive thing on the turn path. Counts answer the health question on their
own.

The existing `catch` keeps the hook fail-open, so a write failure still cannot block a prompt.

`results: 0` is a legitimate row and correctly distinguishes "ran, matched nothing" from "never
ran" — the distinction the health check is trying to make and currently cannot.

## Before / after

| | Before | After |
|---|---|---|
| `memory-retrievals.jsonl` | never created | one row per turn with a prompt |
| `retrieval-missing` on a healthy install | fires forever | clears |
| Retriever ran but matched nothing | indistinguishable from never running | `results: 0`, `totalSearched > 0` |
| Retriever never invoked (the #1573 class of bug) | silent | no rows, warning fires correctly |

## Alternative

If per-turn rows are considered too chatty, the other consistent fix is to drop retrieval from the
health contract and remove the readers. What should not stand is a graded signal with no producer.

Happy to open a PR for either.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in hooks/MemoryTurnStart.hook.ts around the getRelevantContext call at line 121, using MemoryReviewer.ts:61 as the existing JSONL-writing pattern. Run bun LIFEOS/TOOLS/MemoryHealthCheck.ts and verify that a prompt creates memory-retrievals.jsonl, records valid retrieval counts including zero-result runs, and allows retrieval-missing to clear without blocking the hook.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
observability
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.