Coverage disclosure for the external audit behind the recent issues — what was NOT checked, and the specific reason for each

Over the last day I filed a number of issues and advisories from an external audit of v0.44.2 (700bdde). Several of them carry a "not checked" note. This issue collects all of those in one place, with the reason for each, so the gaps are legible rather than scattered across a dozen threads.

The reason for posting this: an audit that reports only what it found invites the reading that everything else was examined and was fine. That is not true here. Below is what I could not examine, and why. "Not checked" means no evidence either way — not "clean".

Provenance: AI-assisted source audit, human-directed. The auditing environment was a single Windows 11 machine with Rust 1.97.1, Node 24, .NET 10 and Git. Nothing else in the list below was installed, for reasons given.

A. Not checked because the tool was not installed — and installing it would not have helped

For these, rtk's filter parses the tool's real output. I used fixtures and shims, which prove the parser's behaviour on the input I supplied but say nothing about what the tool actually emits today. Installing the binary alone would not have closed the gap, because each of these needs credentials, a cluster, a daemon, or a database that I do not have and should not have:

Concretely, this means: every claim I made about an aws, kubectl, psql, docker, terraform, ansible, or sops filter is a claim about rtk's parsing code, not about how that filter behaves against the real tool. Where I said a filter drops a field or mis-parses a format, I verified it in rtk's source and against a fixture I wrote. A maintainer with the real tool can confirm or refute in one run.

B. Not checked because I was on the wrong OS

The audit machine is Windows 11. Not examined:

• #[cfg(unix)] code paths — including the file-permission hardening (set_owner_only is a no-op on non-unix), signal handling, SIGPIPE behaviour, process-group semantics, and orphan reparenting. Several findings I filed reason about Unix behaviour from source; none of it was observed.
• macOS specifics — the Apple Silicon startup-overhead numbers in #1375 are someone else's; my own timings are Windows, where the bare process-spawn floor is ~35 ms, so absolute values are not comparable.
• Linux release artifacts — the musl/gnu split, the .deb/.rpm packages, and the install script's behaviour were read, not run.

C. Not checked because it needs another vendor's product

rtk integrates with several agent harnesses by writing hook configs and emitting JSON they consume. I could not verify the receiving end for:

• GitHub Copilot CLI — whether it honours permissionDecision as rtk supplies it. (Your own #3037 answered part of this; I have no independent view.)
• Gemini CLI, Cursor, Cline/Roo, Windsurf, OpenCode, Droid, JetBrains Copilot plugin — the schemas in hook_cmd.rs are documented in comments and I read them, but I never ran any of these products. If a vendor changed their hook schema, I would not know.
• Claude Code itself — I exercised rtk hook claude directly by piping JSON, which tests rtk's side. I did not run it inside a real Claude Code session.

D. Not checked because it lives outside the repository

• Branch protection and required status checks. Issue #3394 argues two CI steps cannot fail. Whether those jobs are even required for a merge is a GitHub repository setting, not a file in the tree — I cannot see it. If they are not required, the finding is larger than I stated; if some other gate compensates, smaller.
• The telemetry server. Retention, deletion, IP handling, and what the /erasure endpoint actually does are all server-side. Advisory GHSA-q584 reports what the client sends; I make no claim about what happens after.
• crates.io / release-artifact provenance. I did not download and inspect your published binaries (including not running strings on them to look for a compiled-in token — that felt inappropriate for a report you had not yet triaged).

E. Not checked because I chose not to run it

Stated so it is not mistaken for an oversight:

• rtk init and its uninstall paths — these write into the real home directory. I read the code and never executed it.
• rtk cc-economics — it invokes npx --yes and reads real spend data.
• rtk telemetry forget — it deletes the real local database and fires a network request. GHSA-q584 is therefore a code-property report, not an executed one.
• Anything that would have required generating live credentials to make a realistic test case.

F. Read but not audited in depth

Large surfaces I opened enough to navigate, but did not review line by line:

• src/hooks/init.rs (~280 KB) — grepped, not read in full.
• src/discover/registry.rs (~155 KB) — the 178 rewrite rules were counted and the RegexSet path traced; the individual rules were not each validated.
• src/filters/*.toml — 63 declarative filter definitions. At the time I filed the sops advisory I had opened exactly one of them.
• src/openclaw/, src/cmds/php/, most of src/cmds/js/ and src/cmds/dotnet/ beyond the specific files cited in my reports.
• The test suite: I ran it (2520 tests, all passing) but only examined tests adjacent to findings.

What I would ask

If any finding of mine matters to you, the fastest way to settle it is usually the one thing I could not do: run it against the real tool. Where that is the case I said so in the report. I would rather you close something as "works fine against real kubectl" than have it sit open on my say-so.

And if you would find it useful, I am happy to re-check anything on this list against a fixture you consider representative — a captured output file is enough for most of category A.

Per-report ledger: what is still unverified in each thing I filed

The categories above are the reasons. This is the mapping — for every post I made, what specifically remains unverified in it. Read this as: if you are triaging one of these, here is the part you should not take on my word.

Legend: ✅ RAN = I executed it and observed the result · ⚠️ PARTIAL = the mechanism was executed but one step was not · 📖 SOURCE = read from code only, never executed.

Reports sent through the private security channel

Those carry their own "Testing" section stating exactly what was and was not executed — the same discipline as above. I am not restating them here while they are in triage. In short: two were reproduced end-to-end on this machine, one had its mechanism reproduced but not its tool-specific step (the tool is not installed), and two are code-property reports I deliberately did not execute because running them would have fired a real network request or deleted real local data.

The honest summary

Of the 16 public posts above: 5 were executed and observed, 6 had part of the claim executed, and 5 are source-reading only. The five source-only ones are the ones I would most want you to check before acting — they are also, not coincidentally, the ones covering tools I could not install.

Start with the per-report ledger in this issue and the cited entry points, including hook_cmd.rs, src/hooks/init.rs, src/discover/registry.rs, and the relevant filter definitions. Select a specific unverified claim and compare the source-reading evidence with a representative real-tool run or fixture. Done means the selected gap is confirmed or refuted and the result is recorded against the corresponding report.