Security as an intelligence layer, not a byproduct — scoping check before we build

Where this comes from

We run repowise across a workspace of TypeScript/Next.js repos. Digging into the
security surface, we found the plumbing is genuinely good and the analysis
behind it is thin. Before we spend weeks on it, we would like to know which
parts of the following you would take, and which you would rather we keep in a
fork.

What already exists (and works well)

• SecurityScanner + the security_findings table, idempotent across re-runs
  via the (repository_id, file_path, kind, line_number, commit_sha) constraint
• repowise security scan --history with commit provenance for secrets that
  were later removed
• GET /api/repos/{repo_id}/security with severity/path/history filters
• Per-request line verification through check_finding_line, so a finding does
  not silently drift when the file changes
• The findings table on the code-health page

None of that needs replacing. Everything below feeds those same surfaces.

The gap

1. The pattern registry is Python-shaped. Eleven regexes: eval, exec,
pickle.loads, subprocess(shell=True), os.system, hardcoded password/secret,
f-string SQL, concatenated SQL, verify=False, weak hashes. On a Python
codebase that is a reasonable starting set. On a Next.js codebase almost none of
it can fire. There is no pattern for dangerouslySetInnerHTML,
child_process.exec, template-literal SQL, NEXT_PUBLIC_-prefixed secrets,
permissive CORS on mutating handlers, or route handlers with no authorization
check.

Measured on a 55-file Next.js/TypeScript app with 14 API route handlers talking
to SQL Server, the working-tree scan produces one finding — a low on a
test file. Manual review of the same routes finds considerably more, including a
route that writes the client-supplied actor name and timestamp straight into an
audit log.

2. There is no aggregation. security is the only surface that returns a
flat list. health has a 1-10 score with defect-backed bands; risk has an
attributable 0-10 breakdown. Security has row count. There is no way to ask
"is this repo better or worse than last month", and no gate a CI job can fail on.

3. Agents cannot reach it. There is a tool for health, risk, change risk,
blast radius, dead code, architecture and rationale. There is none for security
findings. The UI can see them; the thing the product exists to inform cannot.

4. It is not a layer. docs/layers/ documents every intelligence layer —
health, change risk, dead code, decisions, graph, refactoring, test
intelligence, bug history. Security has no entry, which matches how it is built:
a scan that happens during indexing rather than something reasoned about.

What we propose, in independently mergeable pieces

Tier 1 — TS/JS patterns. Extend the registry so the existing tab is useful
on JavaScript and TypeScript repos. Pure regex plus the existing multi-line
pass, zero new surface, tests in the style of
tests/unit/analysis/test_security_scan.py. This is the piece we would start
with regardless of the answer to the rest.

Tier 2 — get_security MCP tool. A thin adapter over the existing query,
in the same seam get_health and get_risk use.

Tier 3 — Reachability instead of more regexes. health/dataflow/
(reaching, defuse, gating, slice) already does the analysis that
separates "there is an exec call here" from "user input reaches this sink with
no gate in between". Reusing it for sinks and sources would put the security
signal on a footing no grep-based tool has, using machinery that already ships.

Tier 4 — A deterministic score. 1-10 with bands, mirroring grading.py.
We would not ship invented weights. The methodology in scoring.py transfers
directly: label files by whether they later received a security fix (CVE
references, fix(security):, injection/auth/XSS keywords — fix_attribution.py
already does the equivalent for defects), fit offline, ship constants, keep the
runtime pure-deterministic and zero-LLM.

One design constraint we learned the hard way while prototyping: systemic
findings must be grouped before they are scored. Thirteen route handlers missing
an auth check is one architectural decision, not thirteen problems, and a score
that deducts thirteen times reports "everything is broken" permanently and gets
ignored.

The part we expect you to push back on

ROADMAP says you are not building an LLM-based PR reviewer, and gives the
reasons: nothing to hallucinate, nothing to prompt-inject, same diff in, same
review out. We take that seriously, so we want to ask directly rather than
submit something and find out.

What we would like to explore is an opt-in annotation layer on top of the
deterministic findings — the shape health/refactoring/llm/enrich.py already
established: never in the indexing hot path, cached on disk by content hash,
absent entirely when no provider is configured. It would produce triage verdicts,
the exploit path in prose, and a fix diff. It would not produce the score,
and it would not filter the findings the score is computed from. The number
stays reproducible whether or not anyone ever configures a key.

We are aware this may still land on the wrong side of your line, and that is a
legitimate answer.

There is a middle option we would be equally happy with: the "AI fix prompt"
pattern from the code-health page. Assemble the finding, the dataflow path and
the source into a paste-ready prompt and make zero model calls. Same value to
the user, no provider dependency, no hallucination surface, fully within the
existing product stance.

Context for why we care about the model-backed version specifically: security
review is a case where general-purpose models hedge or refuse, and vendors have
started shipping models calibrated for defensive security work. We have access
to one and are measuring it against the deterministic baseline. Whatever comes
of that, we would keep it strictly opt-in and strictly out of the score.

What we are asking

1. Would you take Tier 1 and Tier 2 as PRs?
2. Is Tier 3 (reachability via the existing dataflow machinery) something you
   want in-tree, or is it too close to a SAST product for what repowise is?
3. Is Tier 4 welcome if it arrives with the same calibration rigour as
   CODE_HEALTH.md, or is one defect-backed score enough for this product?
4. On the LLM layer — is the opt-in annotation shape acceptable, is the
   prompt-generator shape the ceiling, or is the whole direction out of scope?

Happy to do the work either way. If some of this belongs in a fork rather than
upstream, we would rather know that now than after the PR.

Start by reading ROADMAP and the existing security scanner, tests/unit/analysis/test_security_scan.py, health/dataflow/, scoring.py, and health/refactoring/llm/enrich.py. The issue spans four proposed tiers and an unresolved LLM policy, so a contributor would first need maintainer direction on one bounded slice; done would require an agreed scope and corresponding tests or design decision.