danielmiessler / danielmiessler/LifeOS
`memory-retrievals.jsonl` is graded by health but never written by anything
- Dominant language
- TypeScript
- Stars
- 19k
- Forks
- 2.5k
- Avg merge
- 8d 17h
- Merged PRs (30d)
- 1
Description
## Summary
`MemoryHealthCheck` grades the memory subsystem on retrieval evidence read from
`LIFEOS/MEMORY/OBSERVABILITY/memory-retrievals.jsonl`. Two tools read that file and the Cortex
contract specifies it. **Nothing in the payload writes it.**
The result is a warning that can never clear on any install. `retrieval-missing` fires on a
healthy system, every day, forever.
Retrieval itself works correctly. Only the logging leg is missing.
- **LifeOS:** 7.40.4
- **Cortex (Memory):** v8.3.0
- **Platform:** Windows 10 Pro, bun 1.3.10
## Evidence
**Readers — 2:**
- `LIFEOS/TOOLS/CortexHealth.ts:206` — `const retrievalLog = join(obs, "memory-retrievals.jsonl"), retrieval = readJsonl(retrievalLog), retrievalLast = retrieval.rows.at(-1);`
- `LIFEOS/TOOLS/MemoryStatus.ts:44` — `const RETRIEVALS_PATH = pathJoin(OBS_DIR, "memory-retrievals.jsonl");`
**Specified in the contract:**
`LIFEOS/DOCUMENTATION/Memory/CortexContract.md:211` —
> Retrieval evidence comes from the latest valid `memory-retrievals.jsonl` row.
Also referenced in `Memory/MemorySystem.md` and `Observability/ObservabilitySystem.md`.
**Writers — 0:**
`LIFEOS/TOOLS/MemoryRetriever.ts` contains no write of any kind: no `writeFile`, no `appendFile`,
no reference to `OBSERVABILITY`. Neither does `hooks/MemoryTurnStart.hook.ts`, its only caller.
A grep for `memory-retrievals` across the install matches five files — the two readers above and
three documentation files.
## Reproduce
On any install, after normal use:
```bash
ls ~/.claude/LIFEOS/MEMORY/OBSERVABILITY/memory-retrievals.jsonl
# No such file or directory
grep -rl "memory-retrievals" ~/.claude/LIFEOS --include=*.ts
# LIFEOS/TOOLS/CortexHealth.ts
# LIFEOS/TOOLS/MemoryStatus.ts <- both read
bun ~/.claude/LIFEOS/TOOLS/MemoryHealthCheck.ts
```
```json
{
"id": "retrieval-missing",
"severity": "warn",
"message": "No retrieval evidence within 86400000ms freshness window.",
"detail": { "status": "missing", "thresholdMs": 86400000 }
}
```
Meanwhile retrieval is demonstrably working — `` blocks render on every turn from
the same code path.
## Why it matters
A permanently-on warning is worse than no warning. It trains operators to read `warn` as the
normal state, which is how the next real finding gets skipped. It also makes the retrieval leg
unobservable: there is no way to tell a retriever returning nothing from a retriever never called,
which is exactly what #1573 turned out to be.
## Proposed fix
Log one row per retrieval at the call site. `getRelevantContext` already returns everything the
readers want, and the write follows the same pattern `MemoryReviewer.ts:61` uses for
`reviewer-runs.jsonl`.
In `hooks/MemoryTurnStart.hook.ts`, around the existing call at line 121:
```diff
if (prompt.trim().length > 0) {
try {
const ground = getRelevantContext(prompt, { topK: 5, threshold: 0.20 });
+ appendFileSync(
+ pathResolve(CLAUDE_ROOT, "LIFEOS/MEMORY/OBSERVABILITY/memory-retrievals.jsonl"),
+ JSON.stringify({
+ ts: new Date().toISOString(),
+ results: ground.results.length,
+ totalSearched: ground.totalSearched,
+ cached: ground.cached,
+ injected: ground.markdownBlock.length > 0,
+ }) + "\n",
+ );
if (ground.markdownBlock) {
process.stdout.write(`\n${ground.markdownBlock}\n\n`);
}
} catch (e) {
process.stderr.write(`MemoryTurnStart ground error: ${(e as Error)?.message || String(e)}\n`);
}
}
```
Deliberately no query text and no result content in the row: this file is diagnostic, and the
prompt is the most sensitive thing on the turn path. Counts answer the health question on their
own.
The existing `catch` keeps the hook fail-open, so a write failure still cannot block a prompt.
`results: 0` is a legitimate row and correctly distinguishes "ran, matched nothing" from "never
ran" — the distinction the health check is trying to make and currently cannot.
## Before / after
| | Before | After |
|---|---|---|
| `memory-retrievals.jsonl` | never created | one row per turn with a prompt |
| `retrieval-missing` on a healthy install | fires forever | clears |
| Retriever ran but matched nothing | indistinguishable from never running | `results: 0`, `totalSearched > 0` |
| Retriever never invoked (the #1573 class of bug) | silent | no rows, warning fires correctly |
## Alternative
If per-turn rows are considered too chatty, the other consistent fix is to drop retrieval from the
health contract and remove the readers. What should not stand is a graded signal with no producer.
Happy to open a PR for either.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in hooks/MemoryTurnStart.hook.ts around the getRelevantContext call at line 121, using MemoryReviewer.ts:61 as the existing JSONL-writing pattern. Run bun LIFEOS/TOOLS/MemoryHealthCheck.ts and verify that a prompt creates memory-retrievals.jsonl, records valid retrieval counts including zero-result runs, and allows retrieval-missing to clear without blocking the hook.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, typescript
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100