[Feature] Bound the cost of an LLM scan: --max-llm-calls / --llm-token-budget, and --estimate before spending

Summary

There is currently no way to bound what an LLM-backed scan will spend before starting it. skillspector scan --llm either completes or does not, and the cost is only knowable afterwards — if at all. I would like a cap that is enforced during the scan, plus a way to see the projected cost before committing to it.

Motivation

Unlike an API-key provider with a prepaid balance, the CLI providers (claude_cli, codex_cli, gemini_cli) draw on the user's personal subscription. There is no per-scan meter, no spending alert, and no natural stopping point — the cost surfaces later as consumed quota.

That asymmetry matters for how SkillSpector gets adopted. A scanner that might cost an unknown amount is one users run rarely and reluctantly; a scanner that can be told "spend at most N calls on this" is one they can wire into a pre-install gate or a timer.

Concretely: I disabled the LLM stage entirely in my automation, not because it lacked value — it caught a natural-language exfiltration skill that the static stage scored 0 findings on — but because I could not bound what an unattended run would cost. The static-only stage I fell back to is precisely the one blind to that attack class. A cap would have let me keep the layer that works.

Scan cost also scales with the unit, not with the user's intent: pointing --changed at a marketplace containing hundreds of skills is one command, and there is no signal beforehand that it is three orders of magnitude more expensive than the last one.

Proposed change

Two independent pieces; the first is the useful half if only one lands.

1. A hard cap, enforced mid-scan

--max-llm-calls N        stop issuing LLM calls after N
--llm-token-budget N     stop after N total tokens (needs #242)

Plus env equivalents (SKILLSPECTOR_MAX_LLM_CALLS, SKILLSPECTOR_LLM_TOKEN_BUDGET) so timers and CI can set them without rewriting the command.

2. A projection, before spending

--estimate               report planned LLM call count and exit without calling

Even a crude count — analyzer slots × eligible components — is enough to distinguish "4 calls" from "400" and is what makes the cap easy to choose.

Behaviour when the cap is hit

This is the part I would ask for care on, because there is a trap adjacent to it that this project already knows about.

A scan stopped by its budget must not be reportable as complete. #290 covers the analogous failure — a deep scan reporting SAFE when the provider was unavailable — and a budget cap creates a second, more frequent path to the same wrong conclusion: an unfinished scan that looks clean because the LLM stage never reached the risky component.

Suggested: mark the report explicitly, e.g. metadata.llm_budget_exhausted: true alongside the existing llm_available / llm_calls_attempted, and have the risk assessment refuse to assert a clean verdict on an unfinished scan. A non-zero exit code for "stopped early" would let CI distinguish it from "finished, nothing found".

Whether the cap should be per-scan or per-skill in recursive mode is a real design question I do not have a strong view on. Per-skill avoids one large skill starving the rest; per-scan is what a user's wallet actually cares about.

Related

• #242 (expose token usage in JSON) is a prerequisite for the token-budget variant. The call-count cap does not depend on it and could ship first.
• #290 (deep scan reports SAFE when provider unavailable) is the failure mode this must not reproduce.
• #295 is a concrete case of cost being unbounded and invisible: 4 attempted LLM calls consumed 3h09m of CPU on my machine, and llm_calls_attempted: 4 in the report was accurate while saying nothing about what had actually been spent.

Start at the skillspector scan --llm CLI entry point and trace how --changed, llm_calls_attempted, and provider availability reach the report. Compare the requested call cap and --estimate behavior with the report requirements, including metadata.llm_budget_exhausted and a distinct outcome when scanning stops early; token budgeting depends on #242.