Make warlock integration harness-agnostic: per-run scanner instance + streaming results
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Problem
warlock's core is already portable — scan(content) -> matches is a pure, stateless engine that drops into any project. But the reporting/telemetry lifecycle currently has to be wired by each integrator, and the wizard's integration (src/lib/yara-hooks.ts) reveals why that's fragile:
- It accumulates scan results into a module-level singleton (
scanViolationsarray) and flushes once at "the end" of a run, then resets. - This has two portability defects:
- It needs an "end." Accumulate-then-flush-once forces every harness to define and correctly hook "the run is over." Miss it and reporting silently never fires.
- It's a singleton. One shared buffer per process silently assumes one sequential run. It breaks under: parallel/fan-out harnesses (runs clobber each other), long-running daemons (never "ends"), and serverless (no shared module state).
The wizard just hit this concretely: when its runner was split into multiple harnesses (linear + experimental orchestrator), the report flush — bolted into one exit path — would silently not fire for the other harness. The current stopgap is an idempotent flushScanReport() finalizer wired at the wizard's single dispatcher seam, but that's wizard-internal glue; it doesn't port to a different harness shape.
Goal
Anyone should be able to hook warlock into any project — whatever their harness shape — with ≤1 line of integration code and no knowledge of warlock's reporting lifecycle.
Proposed shape
The root cause of both defects is batching into a singleton. Remove the batch and both evaporate. Offer two complementary triggers so no harness lifecycle has to be reasoned about:
-
Push (streaming) — removes the "end" problem entirely. A per-run scanner emits each match to a consumer-supplied sink as it happens:
const scanner = warlock.createScanner({ rules, onMatch: m => sink.send(m) });No buffer, no "end" — daemon, serverless, and parallel fan-out are all identical.
-
Per-run instance + dispose — for consumers who still want one aggregated summary. The buffer moves off the module singleton and into a scanner instance, flushed on dispose (scope-bound, concurrency-safe):
await using scanner = warlock.createScanner({ rules }); // owns its own results // ... summary auto-emitted at scope exit, or scanner.report() on demand
Between push (no end needed) and dispose (scope-bound end), virtually every harness is covered with ≤1 line; the exotic case falls back to explicit scanner.close().
Layering (respects warlock's engine-only philosophy)
| Layer | Owns | Ports? |
|---|---|---|
| warlock core | scan -> matches (pure) |
anywhere, already |
| thin optional adapter | per-run scanner instance: streaming onMatch + disposable summary |
anywhere — no harness assumptions |
| integrator glue | maps the adapter onto this harness's hooks | tiny, per-project |
The consumer still decides how to respond to matches (warlock stays engine-only); this just stops every integrator from re-implementing accumulation + lifecycle.
Acceptance
- No module-level singleton state in the reporting path.
- A new integrator can wire warlock into a fresh project's harness (any shape) with ≤1 line and no lifecycle knowledge.
- Concurrent/parallel runs don't share or clobber scan state.
Context: surfaced while integrating warlock into the PostHog wizard (PostHog/wizard, yara-improvements branch).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/lib/yara-hooks.ts and trace how the module-level scanViolations buffer and flushScanReport() are used. Use the proposed createScanner entry point and acceptance criteria to define the adapter boundary, then verify that streaming and per-run summaries do not share state across concurrent runs and that a new harness needs no lifecycle-specific wiring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- security, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100