danielmiessler / danielmiessler/LifeOS
Doctor's seven binary capabilities check PATH presence, not liveness — a broken ffmpeg and a 14-month-old yt-dlp both reported "live" while failing every call
- Dominant language
- TypeScript
- Stars
- 19k
- Forks
- 2.5k
- Avg merge
- 8d 17h
- Merged PRs (30d)
- 1
Description
### Version
LifeOS 7.40.4 / Doctor
### What is broken
Seven capabilities in `Doctor.ts` — ripgrep, ImageMagick, gh, ffmpeg, yt-dlp, fabric, jq — resolve their state with `which(bin)`, which tests only whether a file of that name exists on `PATH`. That answers "is something installed under this name", not "does this tool work". Two distinct real failures pass it:
1. **A binary that cannot execute.** An ffmpeg whose linked library moved (here: `libx265.215.dylib`, after an x265 upgrade left the formula unrelinked) aborts on every invocation with a dyld error and exit 134. `which('ffmpeg')` is true throughout, so Doctor reports the capability live while AudioEditor, transcript splitting and the Interceptor zoom fallback fail at spawn.
2. **A binary that runs but has rotted against a moving target.** yt-dlp is date-versioned and YouTube breaks old builds outright. A build from 2025-06-30 executed fine and failed every URL with "The following content is not available on this app". Doctor reported it live for the entire period.
This is the appearance-versus-existence distinction the project's own verification doctrine forbids elsewhere — a green check over a dead dependency — committed by the checker whose job is to catch it. The cost is asymmetric: a user reads the green line and trusts the capability, so the failure surfaces later as an unexplained tool error rather than at the point where Doctor could have named it.
### Where (file:line)
`LIFEOS/TOOLS/Doctor.ts:109` — `function which(bin: string): boolean` (existsSync on PATH only)
Consumers: `Doctor.ts:473` (rg), `:485` (magick), `:497` (gh), `:509` (ffmpeg), `:521` (yt-dlp), `:533` (fabric), `:545` (jq).
### Repro on a clean tree
```shell
# Reproduce case 1 — a present but unrunnable binary.
# Any binary on PATH that aborts will do; a broken dylib link is the natural case.
ffmpeg -version # => dyld: Library not loaded: .../libx265.215.dylib ; exit 134
bun LIFEOS/TOOLS/Doctor.ts | grep -i ffmpeg
# => ✅ Audio/video processing (ffmpeg) — live
# => ffmpeg on PATH
# Reproduce case 2 — a present but rotted binary.
# With yt-dlp 2025.06.30 installed:
yt-dlp --skip-download --write-auto-sub
# => ERROR: The following content is not available on this app.
bun LIFEOS/TOOLS/Doctor.ts | grep -i yt-dlp
# => ✅ YouTube ingestion (yt-dlp) — live
# => yt-dlp on PATH
```
### Negative control
On unpatched 7.40.4, Doctor reported 14 capabilities with one broken (Interceptor, unconfigured) and zero broken among the seven PATH-checked tools, while two of those seven were in fact non-functional: `ffmpeg -version` exited 134 on every call, and yt-dlp failed every YouTube URL. The check has therefore never been red for either failure mode. Replacing `which()` with an invocation that requires exit 0 turned ffmpeg red on the first run, before anything was known to be wrong with it — the dyld failure was discovered by the check, not the other way round. Whole-run cost after the change: 1.1s wall clock for all 14 capabilities.
### Suggested fix
Keep the fast absence path, then actually run the tool. Sketch, tested locally:
```ts
async function probeLive(opts: {
bin: string; versionArgs?: string[]; absentDetail: string; staleAfterDays?: number;
}): Promise<{ ok: boolean; detail: string }> {
const { bin, versionArgs = ['--version'], absentDetail, staleAfterDays } = opts;
if (!which(bin)) return { ok: false, detail: `${bin} not on PATH — ${absentDetail}` };
const { code, out } = await run([bin, ...versionArgs], 4000);
if (code === 124) return { ok: false, detail: `${bin} on PATH but \`${bin} ${versionArgs.join(' ')}\` timed out — the binary is wedged` };
if (code !== 0) return { ok: false, detail: `${bin} on PATH but \`${bin} ${versionArgs.join(' ')}\` exited ${code} — installed and not runnable` };
const version = (out.split('\n')[0] || '').trim() || 'version unreported';
if (staleAfterDays) {
const m = version.match(/(\d{4})\.(\d{2})\.(\d{2})/);
if (m) {
const ageDays = Math.floor((Date.now() - Date.UTC(+m[1], +m[2] - 1, +m[3])) / 86400000);
if (ageDays > staleAfterDays) return { ok: false, detail: `${bin} runs but is ${ageDays} days old (${version}) — date-versioned tools rot against the services they scrape` };
return { ok: true, detail: `${bin} runs (${version}, ${ageDays}d old)` };
}
}
return { ok: true, detail: `${bin} runs (${version})` };
}
```
Two notes from using it. ffmpeg needs `versionArgs: ['-version']` — a single dash; it is not a GNU-style CLI and aborts with 134 on `--version`, which the check caught on its own first run. And `staleAfterDays` is worth setting only where age is a real health signal: yt-dlp yes, rg and jq no.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in LIFEOS/TOOLS/Doctor.ts at the which function and the seven capability checks at lines 473–545, then run the documented ffmpeg and yt-dlp reproductions. Trace the existing run helper and confirm how Doctor reports capability details. Done means absent, non-executable, timed-out, and stale date-versioned tools are distinguished while healthy tools report their versions and the full check remains responsive.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, typescript
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100