dequelabs / dequelabs/axe-api-team-public

Migrate repo tooling from npm workspaces to pnpm

Open
#421 0 comments 0 reactions 0 assignees View on GitHub
PRIORITY: moderate SIZE: medium tech-debt TYPE: tech-debt
Dominant language
JavaScript
Stars
0
Forks
1
Avg merge
16h 50m
Merged PRs (30d)
2

Description

## Summary

Migrate this repo's dev/build tooling from npm workspaces to pnpm. Benefits are dev-facing only: faster, content-addressed installs, meaningful disk savings across the workspace store, and stricter dependency resolution that surfaces phantom deps. This is internal tooling churn with **no consumer-facing impact** — downstream repos load the committed `dist/index.js`, which we regenerate and re-commit as part of the migration. The one real catch is that pnpm's default `node_modules` layout differs from npm's, which we pin back to a flat layout so the build pipeline keeps working (see Key risk below).

## Background — current setup

- **Workspaces:** root `package.json` has `"workspaces": ["./.github/actions/*"]`. That glob spans **31 directories**, but only **20 have a `package.json`** and are real build workspaces; the other 11 are composite/YAML-only actions with no build and no `dist` from this pipeline. The migration builds **20 workspaces**, not 31.
- **Deps live only at the root:** every third-party dep (`@actions/*`, `esbuild`, `glob`, `semver`, `ignore`, `dedent`, `conventional-*`, eslint/prettier/typescript/tsx/rimraf/husky/lint-staged, etc.) is declared in root `devDependencies`. Action workspaces declare **no deps** — with one exception: `checkov-scans-v1` declares its own `pnpm-lock-to-npm-lock@1.0.0`. Actions rely 100% on npm's flat hoisting into a single root `node_modules`.
- **Build:** `scripts/build-action.mjs` runs esbuild **from inside each action dir** (`node ../../../scripts/build-action.mjs`), bundling `src/index.ts` → `dist/index.js` and walking `node_modules` to write `dist/licenses.txt`. Both `dist/` files are **committed** and are what the Actions runtime loads.
- **CI:** `tests.yml` is **14 hand-written near-duplicate jobs** (not a matrix), each doing `actions/checkout` + `actions/setup-node@v6` (`cache: 'npm'`) + `npm ci` + one lint/typecheck/test command. `update-generated-files.yml` does `npm ci` + `npm run build -ws`, then re-commits rebuilt `dist/` via a `sync-generated-files` PR.
- **Husky:** `.husky/build-changed-files.sh` runs `npm run build -w "$dir"` for each changed workspace dir; `.husky/pre-commit` runs `npx lint-staged`.
- **Dependabot:** `.github/dependabot.yml` has `github-actions` + `npm` ecosystems (`directory: '/'`, monthly, grouped `eslint` + `npm-low-risk`, 7-day cooldown).
- **State today:** no `.npmrc`, no `packageManager` field, `package-lock.json` is the committed lockfile. `.gitignore` ignores `node_modules`, `coverage`, `*.log`, `yarn.lock`.

## Key risk: node_modules layout

**This is the crux of the migration.** pnpm's default `node-linker` is `isolated` — each package gets a `node_modules` containing only its own declared deps, plus a symlinked `.pnpm` virtual store. Our 20 action workspaces declare no deps, and `build-action.mjs` runs esbuild from inside each action dir and relies on Node's upward `node_modules` walk to find root-level deps. Under isolated linking this technically still resolves (direct root deps appear at root, walk-up finds them), **but the committed `dist/` changes materially**: esbuild inlines `node_modules`-relative paths into `dist/index.js` (as banner comments and CommonJS registry keys), so under isolated linking every path gains a long `.pnpm/@/node_modules/` prefix and duplicate versions get bundled (empirically: `add-to-board-v1/dist/index.js` grew +8,755 bytes and bundled **two** copies of `@actions/http-client`). That is a large, dual-version diff shipped to consumers.

**Recommended fix (mandatory, not optional): pin the flat layout with `.npmrc`.** Create `.npmrc` at the repo root:

```
node-linker=hoisted
```

This reproduces npm's flat top-level `node_modules`, keeps esbuild's embedded paths short, keeps the license walker's discovered paths shortest, and lets any phantom/transitive import still resolve. It is **not** a byte-identity guarantee — even under `hoisted`, ~19 of 20 `dist/index.js` shift by a handful of bytes because `@actions/http-client` legitimately exists at two versions (`^3.0.2` and `^4.0.0`) and pnpm's hoist-winner differs from npm's, so the embedded path strings differ. `licenses.txt` is unaffected (it emits name/license/text only, never paths). **Conclusion: dist regeneration is mandatory** — we rebuild all 20 workspaces under pnpm and commit that as the new baseline in the same PR.

**Cleaner alternative (defer):** declare each action's real deps in its own `package.json`, remove them from root (keep only shared tooling at root), and run under pnpm's default isolated linker with no `.npmrc`. This yields true per-action dependency hygiene and catches phantom deps, but requires auditing/editing ~20 action manifests, risks omitting a dep, and fights the repo's documented "install shared deps at the root" convention. Higher effort/churn — not part of this ticket.

## Migration checklist

Do all of this on one branch, in one PR.

**Config files**
- [ ] Create `pnpm-workspace.yaml` at repo root (pnpm ignores `package.json` `"workspaces"`; the glob matches all 31 dirs but only the 20 with a `package.json` register as workspaces, same as npm today):
```yaml
packages:
- '.github/actions/*'
onlyBuiltDependencies:
- esbuild
```
`onlyBuiltDependencies` is required because pnpm 10 blocks dependency lifecycle scripts by default (install prints `Ignored build scripts: esbuild`). Run `pnpm approve-builds` once to confirm the full set and pin whatever it surfaces.
- [ ] Create `.npmrc` at repo root with a single line: `node-linker=hoisted` (see Key risk).
- [ ] Add `"packageManager": "pnpm@10.x.x"` to root `package.json` (pin the **exact** version you install — this plan was validated against `pnpm@10.33.4`). Use the **plain** version, no `+sha512...` hash: the hash is only needed for corepack, and `pnpm/action-setup` has had edge-case parsing of the hash suffix. `pnpm/action-setup@v4` reads this field for its version.
- [ ] Remove the now-dead `"workspaces": ["./.github/actions/*"]` array from root `package.json` (superseded by `pnpm-workspace.yaml`; pnpm ignores it, so this is cleanup).

**Lockfile**
- [ ] `pnpm import` **first** — this reads the existing `package-lock.json` and seeds a version-identical `pnpm-lock.yaml`, so a fresh `pnpm install` does not re-resolve `^` ranges to newer patches. A plain `pnpm install` would re-resolve and change bundled dependency **versions**, churning both `dist/index.js` and `dist/licenses.txt` beyond the benign path diff.
- [ ] `git rm package-lock.json`.
- [ ] `pnpm install`, then commit `pnpm-lock.yaml`. Confirm no `Ignored build scripts` warning names a package we depend on.
- [ ] `.gitignore`: add `package-lock.json` (so a stray `npm install` can't resurrect it). Do **not** add `pnpm-lock.yaml` (it must be tracked). Do **not** add `.pnpm-debug.log` — `.gitignore` already has `*.log`.

**Root scripts (`package.json`)**
- [ ] `"build": "npm --workspaces run build"` → `"build": "pnpm -r run build"`
- [ ] `"test": "npm --workspaces run test"` → `"test": "pnpm -r run test"`
- [ ] `"typecheck": "npm --workspaces run typecheck --if-present"` → `"typecheck": "pnpm -r run typecheck"` (`pnpm -r` already skips packages lacking the script, so `--if-present` is redundant).
- [ ] Leave `"prepare": "husky install"` as-is (pnpm runs the root `prepare` on install; husky bootstrap keeps working). The husky v9→v10 modernization (`husky install` → `husky`, dropping the `_/husky.sh` preamble) is pre-existing tech-debt — do it here only if you want to, it's not required by pnpm.

**Husky**
- [ ] `.husky/build-changed-files.sh`: `npm run build -w "$dir"` → `pnpm --filter "$dir" run build`. Note pnpm's `-w` means `--workspace-root` (opposite of npm's `-w`), so you **must** use `--filter`. `$dir` equals the package name for all 20 workspaces, so name-filtering works.
- [ ] **Guard the non-workspace dirs:** `build-changed-files.sh` derives dir names from *all* changed files under `.github/actions/`, including the 11 dirs with no `package.json`. Editing a file in one of those yields a `--filter` that matches zero projects, and pnpm's no-match behavior differs from npm's. Add a guard so it only builds dirs that have a `package.json`, e.g. inside the loop: `[ -f "$dir/package.json" ] || continue`. Verify the resulting exit code on a scratch commit touching a non-workspace dir.
- [ ] `.husky/pre-commit`: `npx lint-staged` → `pnpm exec lint-staged` (lint-staged is a root devDependency; `pnpm exec` runs the local binary, `pnpm dlx` would wrongly re-download).

**CI workflows**
- [ ] `tests.yml` — apply to **all 14 jobs**: insert `pnpm/action-setup` **before** `actions/setup-node` (ordering is load-bearing — `cache: 'pnpm'` shells out to `pnpm store path` during setup-node's main step and fails if pnpm isn't on PATH yet), switch `cache: 'npm'` → `cache: 'pnpm'`, `npm ci` → `pnpm install --frozen-lockfile`. Command translations: `npm run lint -ws` → `pnpm -r run lint`; `npm run typecheck` → `pnpm run typecheck`; each `npm run test --workspace=` → `pnpm --filter= run test`. Pin `pnpm/action-setup` by full commit SHA + `# v4.x.x` comment (repo convention — look up the real SHA, do not fabricate). Representative job:

Before:
```yaml
test-semantic-pr-footer-v1:
needs: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24
cache: 'npm'
- run: npm ci
- run: npm run test --workspace=semantic-pr-footer-v1
```
After:
```yaml
test-semantic-pr-footer-v1:
needs: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: pnpm/action-setup@ # v4.x.x
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24
cache: 'pnpm'
- run: pnpm install --frozen-lockfile
- run: pnpm --filter=semantic-pr-footer-v1 run test
```
- [ ] `update-generated-files.yml`: add `pnpm/action-setup` before setup-node; `cache: 'npm'` → `cache: 'pnpm'`; `npm ci` → `pnpm install --frozen-lockfile` (frozen, so a re-resolved lockfile is never silently committed into the sync PR via the `git add .` step); `npm run build -ws` → `pnpm -r run build`; the later `npm run build` → `pnpm run build`. The deprecated `::set-output` lines here are unrelated to pnpm — leave them or clean up to `$GITHUB_OUTPUT` separately.

**Dependabot**
- [ ] `.github/dependabot.yml` — **no change needed.** There is no separate pnpm ecosystem; `package-ecosystem: 'npm'` reads `pnpm-lock.yaml`. Keep `directory: '/'`, the `eslint` + `npm-low-risk` groups, cooldown, and the `github-actions` block. Just confirm the generated lockfile version is one Dependabot supports (pnpm 9/10 lockfiles are).

**README**
- [ ] Keep the "install shared 3rd-party deps at the root `package.json`" rule. Change the alias example (line 9) from `npm install pkg-v1@npm:pkg@1` to `pnpm add -w -D pkg-v1@npm:pkg@1` (root install in a pnpm workspace requires `-w`; `-D` matches the devDependencies convention).

**Verify / regenerate baseline**
- [ ] Rebuild and commit the new dist baseline in this PR: `pnpm -r run build` (optionally `--workspace-concurrency=1` for a cleaner, deterministic diff). Confirm the diff is **path-string / module-key churn only** on `dist/index.js` and that `dist/licenses.txt` is unchanged. This regenerated `dist/` is the new source of truth — the `git diff --exit-code` guard below is only meaningful against it, never against the old npm-built dist.
- [ ] Fix the stale comment in `scripts/build-action.mjs` line 2 (`Run from inside a workspace dir (npm run build)`) to reference pnpm.
- [ ] Canary: run `pnpm --filter checkov-scans-v1 run build` and its test — it's the only workspace with a real declared dep (`pnpm-lock-to-npm-lock`, which pulls a transitive `yaml`), so it validates that `pnpm-workspace.yaml` installs workspace-declared deps and esbuild still bundles the nested/transitive dep.

## Acceptance criteria

- [ ] Fresh clone + `corepack enable` (or pnpm on PATH) + `pnpm install --frozen-lockfile` succeeds, lockfile in sync, no relevant `Ignored build scripts` warning.
- [ ] `pnpm -r run build` followed by `git diff --exit-code -- '.github/actions/*/dist/'` is clean — **against the regenerated pnpm baseline committed in this PR** (dist is reproducible going forward). This is the primary correctness gate protecting downstream consumers.
- [ ] `pnpm -r run test`, `pnpm -r run typecheck`, and `pnpm -r run lint` all pass locally, and all 14 `tests.yml` jobs are green.
- [ ] `update-generated-files.yml` runs green and does **not** open a spurious `sync-generated-files` PR (because the pnpm-built dist is already committed here).
- [ ] Husky pre-commit still builds only changed workspaces and runs lint-staged: make a trivial `src` change in one action, commit, confirm only that action's `dist` rebuilds and is staged; and confirm committing a change in a non-workspace dir doesn't error.
- [ ] Dependabot still opens grouped PRs (`eslint`, `npm-low-risk`) against `pnpm-lock.yaml` (verify on next run or by config review).
- [ ] No `package-lock.json` remains; `pnpm-lock.yaml` committed; `node_modules` still ignored.

## Out of scope

- **`update-axe-core-v1` consumer PM detection:** that action detects a *consumer* repo's package manager (npm/yarn/pnpm) at runtime to run installs in *other* repos. It is unrelated to which PM **this** repo uses for its own dev/build. Do not touch it and do not conflate it with this migration.
- **Downstream consumers:** other repos consume these actions via `uses: dequelabs/axe-api-team-public/.github/actions/@ref` and load the committed `dist/index.js`. As long as we regenerate and commit `dist/` here, consumers need zero coordination or changes. No comms required.
- **Node version bump, esbuild strategy/target (`node24`), removal of the committed-dist workflow, per-action dependency restructuring** (the "cleaner alternative" above) — all out of scope.
- **`::set-output` → `$GITHUB_OUTPUT` cleanup** in `update-generated-files.yml` and **husky v9→v10 modernization** — pre-existing tech-debt, optional side cleanups, not required by pnpm.

## Risks & rollback

- **dist byte-drift → consumer artifact change.** If `.npmrc node-linker=hoisted` is omitted anywhere (including a fresh CI checkout without it committed), pnpm's isolated linker balloons dist and bundles duplicate versions. Mitigation: commit `.npmrc`; gate on the `git diff --exit-code` dist check.
- **First-run dist diff mistaken for a regression.** The pnpm baseline differs from the npm baseline (path-string churn); reviewers/CI comparing against the old dist will see a non-empty diff by design. Call this out in the PR description; the reproducibility guard is only meaningful against the new baseline.
- **esbuild build script blocked by pnpm 10.** Mitigate with `onlyBuiltDependencies: [esbuild]` and confirm via `pnpm approve-builds`; sanity-check the bundle builds on the Linux CI runner.
- **CI ordering trap.** `pnpm/action-setup` must precede `actions/setup-node` in every one of the 14 jobs, or `cache: 'pnpm'` errors with "Unable to locate executable file: pnpm". 14 near-identical edits = real drift risk; a matrix/reusable workflow would collapse this to one (worth doing while touching every job anyway).
- **`pnpm -w` vs npm `-w` inversion.** A mechanical find/replace keeping `-w` would silently run in the workspace root. Use `--filter`/`-r`.
- **Lockfile fidelity.** Use `pnpm import` (not a bare `pnpm install`) to preserve exact versions, or dist/licenses churn beyond path strings.
- **Dependabot lockfile churn.** Dependabot rewrites `pnpm-lock.yaml` with its own internal pnpm; a major mismatch vs the pinned `packageManager` can cause recurring format drift. Pinning mitigates but isn't a hard guarantee.
- **Rollback:** fully reversible in one revert — restore `package-lock.json` from history, delete `pnpm-lock.yaml`/`pnpm-workspace.yaml`/`.npmrc`, drop the `packageManager` field, revert scripts + restore the `workspaces` array, revert both workflows and the husky hook, then `npm ci`. Because the migration leaves `dist/` functionally identical, reverting the tooling has zero effect on consumers.

## Open questions / decisions for the implementer

- [ ] Exact pnpm version to pin in `packageManager` — plan was validated against pnpm@10.33.4; confirm the latest 10.x is acceptable or pin 10.33.4.
- [ ] Which `pnpm/action-setup@v4` commit SHA to pin (repo convention is full-SHA + `# v4.x.x` comment) — needs looking up, not fabricating.
- [ ] Collapse the 14 near-duplicate `tests.yml` jobs into a matrix / reusable workflow as part of this migration (recommended, since every job is edited anyway) or defer to a separate PR?
- [ ] Do the optional husky v9→v10 modernization and the `::set-output` → `$GITHUB_OUTPUT` cleanup in this PR, or keep them separate?
- [ ] Confirm reviewers accept the one-time path-string dist diff as the new committed baseline (vs. expecting byte-identical dist).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with package.json, the new pnpm-workspace.yaml and .npmrc, then inspect .husky/build-changed-files.sh, .husky/pre-commit, .github/workflows/tests.yml, and update-generated-files.yml. Run the documented pnpm install, checks, and build commands, then verify all 20 workspaces rebuild, generated dist/index.js contains only expected path or module-key churn, and dist/licenses.txt is unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, javascript
Domain
build-system, ci-cd, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.