BOHICA-LABS / BOHICA-LABS/vsdd-factory

meta(cicd): unbounded CI-job proliferation — every adversarial finding becomes a new lint job; cost/maintenance arc untracked

Open
#308 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Class of concern (meta / cost-velocity)

VSDD's adversarial+convergence loops are very good at surfacing spec/process defects, and the factory's natural response to many of them is **"add a CI lint job that guards against recurrence."** That reflex is individually correct but, compounded over a pipeline, appears to produce an unbounded, monotonically-growing CI surface whose cost and maintenance arc nobody is explicitly tracking. We don't fully understand the long-run impact yet — this issue asks the framework maintainers to study it, and contributes concrete metrics from one pilot.

## Observed arc (external pilot, ftc-blue — Godot/GDScript)

`ci.yml` growth on a single pilot, from the git history of `.github/workflows/ci.yml`:

| Date | Named jobs/steps | ci.yml lines | Driver (commit subjects) |
|------|------------------|--------------|--------------------------|
| 2026-06-22 | 6 | 374 | initial Phase-1 CI |
| 2026-06-22 | 8 → 14 | 504 → 726 | commentary-coverage gate, era-shader + 5 spec-drift lints |
| 2026-06-23 | 17 → 30 | 831 → 1407 | per-pass process-gap lints wired in pass-3/5/8/9 |
| 2026-06-23 | 33 → 37 | 1519 → 1679 | pass-10/11 lints (citation staleness, enum-anchor, changelog-order, …) |
| 2026-06-24 | **39** | **1777** | GUT import-step + positive-coverage gate |

**~2 days: 6 → 39 jobs (6.5×), 374 → 1777 lines (4.75×).** Almost every addition is traceable to an adversarial-pass finding ("pass-N F-XX") being converted into a permanent guard. The growth has no decay term — nothing ever consolidates or retires a lint.

Of the 39 jobs on this pilot:
- ~30 are **spec-governance lints** (changelog order, citation staleness, namespace collisions, label sync, frontmatter coverage, index-vs-filesystem, …) — they validate `.factory/` artifacts, not product code.
- **33 of 39 jobs independently `git fetch origin factory-artifacts --depth=1`** and re-checkout the orphan branch — the same setup duplicated 33×.
- 5 jobs install Godot.

## The cost signature (this is the part that compounds quietly)

Per-job wall-clock on the most recent run is tiny — **35 of 39 jobs finish in 5–9 s**; the longest (GUT) is 23 s; total wall-clock ≈ 25 s with parallelism. But:

- **GitHub-hosted runners bill each job rounded UP to the nearest minute.** 39 jobs ⇒ **~39 billed job-minutes per CI run**, against ~25 s of actual work — roughly a **90× inflation** of billed minutes vs. wall-clock, driven purely by job *granularity*.
- Each job pays full fixed overhead every run: VM provisioning + `actions/checkout` + `git fetch factory-artifacts` + (for 5) Godot install. The 5–9 s "work" is dwarfed by per-job setup.
- This pilot has executed **175 workflow runs** to date and is only at end-of-Wave-1.

So the trend isn't visible as slow CI (it's fast, parallel), it's visible as **billed-minute and maintenance growth that scales with the number of guards, not the amount of verification work.** Each new adversarial finding nudges both curves up and nothing pushes them down.

## Why this may be a framework-level problem, not a pilot config choice

The convert-finding-to-CI-lint reflex is encoded in the pipeline's DNA (per-pass "wire lint job" commits, `cicd-setup` scaffolding, the cycle-closing codification rule). Any project run through VSDD will likely exhibit the same monotonic CI growth. We don't yet know the steady-state ceiling, the maintenance burden when a spec convention changes (N lint scripts to update), or the false-positive rate as lints accrete.

## Requests

1. **Study the arc across several VSDD projects**, not just this pilot — collect (jobs over time, ci.yml LOC, billed minutes/run, fraction that are spec-governance lints, lint-churn when conventions change) and characterize whether growth is bounded.
2. Consider framework guidance / mechanisms to counter unbounded growth, e.g.:
- **Batch the spec-governance lints into one job** (a single `lint-governance` matrix or one script runner) instead of 30 separate billable jobs — would collapse ~30 job-minutes to ~1–2 and de-duplicate the 33× factory-artifacts fetch.
- A **lint-retirement / consolidation review** at wave-close (does each guard still earn its slot?).
- A **"prefer one parametrized check over N micro-jobs"** principle in `cicd-setup`.
- Track CI job-count + billed minutes as a first-class factory metric with a soft ceiling (mirrors STATE.md size governance).

## Metrics appendix (ftc-blue, single pilot)
- Jobs in ci.yml: **39**; pr-validation.yml: 4. ci.yml lines: **1777**.
- Spec-governance lint jobs: ~30/39. Jobs re-fetching factory-artifacts: **33/39**. Godot installs: 5.
- Per-job wall-clock: 35/39 in 5–9 s; max 23 s. Est. billed job-minutes/run: **~39** (≈90× wall-clock).
- Total workflow runs on repo: **175** (end of Wave 1).

(No product/game-design content disclosed — all figures are framework-pattern metrics.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.