danielmiessler / danielmiessler/LifeOS

Doctor's seven binary capabilities check PATH presence, not liveness — a broken ffmpeg and a 14-month-old yt-dlp both reported "live" while failing every call

Open
#2,066 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
19k
Forks
2.5k
Avg merge
8d 17h
Merged PRs (30d)
1

Description

### Version
LifeOS 7.40.4 / Doctor

### What is broken
Seven capabilities in `Doctor.ts` — ripgrep, ImageMagick, gh, ffmpeg, yt-dlp, fabric, jq — resolve their state with `which(bin)`, which tests only whether a file of that name exists on `PATH`. That answers "is something installed under this name", not "does this tool work". Two distinct real failures pass it:

1. **A binary that cannot execute.** An ffmpeg whose linked library moved (here: `libx265.215.dylib`, after an x265 upgrade left the formula unrelinked) aborts on every invocation with a dyld error and exit 134. `which('ffmpeg')` is true throughout, so Doctor reports the capability live while AudioEditor, transcript splitting and the Interceptor zoom fallback fail at spawn.

2. **A binary that runs but has rotted against a moving target.** yt-dlp is date-versioned and YouTube breaks old builds outright. A build from 2025-06-30 executed fine and failed every URL with "The following content is not available on this app". Doctor reported it live for the entire period.

This is the appearance-versus-existence distinction the project's own verification doctrine forbids elsewhere — a green check over a dead dependency — committed by the checker whose job is to catch it. The cost is asymmetric: a user reads the green line and trusts the capability, so the failure surfaces later as an unexplained tool error rather than at the point where Doctor could have named it.

### Where (file:line)
`LIFEOS/TOOLS/Doctor.ts:109` — `function which(bin: string): boolean` (existsSync on PATH only)

Consumers: `Doctor.ts:473` (rg), `:485` (magick), `:497` (gh), `:509` (ffmpeg), `:521` (yt-dlp), `:533` (fabric), `:545` (jq).

### Repro on a clean tree
```shell
# Reproduce case 1 — a present but unrunnable binary.
# Any binary on PATH that aborts will do; a broken dylib link is the natural case.
ffmpeg -version # => dyld: Library not loaded: .../libx265.215.dylib ; exit 134
bun LIFEOS/TOOLS/Doctor.ts | grep -i ffmpeg
# => ✅ Audio/video processing (ffmpeg) — live
# => ffmpeg on PATH

# Reproduce case 2 — a present but rotted binary.
# With yt-dlp 2025.06.30 installed:
yt-dlp --skip-download --write-auto-sub
# => ERROR: The following content is not available on this app.
bun LIFEOS/TOOLS/Doctor.ts | grep -i yt-dlp
# => ✅ YouTube ingestion (yt-dlp) — live
# => yt-dlp on PATH
```

### Negative control
On unpatched 7.40.4, Doctor reported 14 capabilities with one broken (Interceptor, unconfigured) and zero broken among the seven PATH-checked tools, while two of those seven were in fact non-functional: `ffmpeg -version` exited 134 on every call, and yt-dlp failed every YouTube URL. The check has therefore never been red for either failure mode. Replacing `which()` with an invocation that requires exit 0 turned ffmpeg red on the first run, before anything was known to be wrong with it — the dyld failure was discovered by the check, not the other way round. Whole-run cost after the change: 1.1s wall clock for all 14 capabilities.

### Suggested fix
Keep the fast absence path, then actually run the tool. Sketch, tested locally:

```ts
async function probeLive(opts: {
bin: string; versionArgs?: string[]; absentDetail: string; staleAfterDays?: number;
}): Promise<{ ok: boolean; detail: string }> {
const { bin, versionArgs = ['--version'], absentDetail, staleAfterDays } = opts;
if (!which(bin)) return { ok: false, detail: `${bin} not on PATH — ${absentDetail}` };

const { code, out } = await run([bin, ...versionArgs], 4000);
if (code === 124) return { ok: false, detail: `${bin} on PATH but \`${bin} ${versionArgs.join(' ')}\` timed out — the binary is wedged` };
if (code !== 0) return { ok: false, detail: `${bin} on PATH but \`${bin} ${versionArgs.join(' ')}\` exited ${code} — installed and not runnable` };

const version = (out.split('\n')[0] || '').trim() || 'version unreported';
if (staleAfterDays) {
const m = version.match(/(\d{4})\.(\d{2})\.(\d{2})/);
if (m) {
const ageDays = Math.floor((Date.now() - Date.UTC(+m[1], +m[2] - 1, +m[3])) / 86400000);
if (ageDays > staleAfterDays) return { ok: false, detail: `${bin} runs but is ${ageDays} days old (${version}) — date-versioned tools rot against the services they scrape` };
return { ok: true, detail: `${bin} runs (${version}, ${ageDays}d old)` };
}
}
return { ok: true, detail: `${bin} runs (${version})` };
}
```

Two notes from using it. ffmpeg needs `versionArgs: ['-version']` — a single dash; it is not a GNU-style CLI and aborts with 134 on `--version`, which the check caught on its own first run. And `staleAfterDays` is worth setting only where age is a real health signal: yt-dlp yes, rg and jq no.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in LIFEOS/TOOLS/Doctor.ts at the which function and the seven capability checks at lines 473–545, then run the documented ffmpeg and yt-dlp reproductions. Trace the existing run helper and confirm how Doctor reports capability details. Done means absent, non-executable, timed-out, and stale date-versioned tools are distinguished while healthy tools report their versions and the full check remains responsive.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.