feat: automated release train — version enforcement, reusable rule workflows, and publish automation
- Dominant language
- Shell
- Stars
- 0
- Forks
- 1
- Avg merge
- 12h 40m
- Merged PRs (30d)
- 2
Description
## Summary
This issue tracks the full design and implementation of an automated release pipeline for all `frmscoe` rule repos. The work is the `frmscoe`-org counterpart to [tazama-lf/workflows#30](https://github.com/tazama-lf/workflows/issues/30) and extends the canonical improvements previously identified in #65.
It covers four interconnected areas:
1. **`publish.yml`** — full replacement of the old-generation file (Node 16, `checkout@v3`, auto-versioning, token leak)
2. **`version-check.yml`** — new gate workflow that blocks PRs to `main` if `package.json` still contains a prerelease version suffix
3. **`release-train.yml`** — new `workflow_dispatch` automation to prepare and open a release PR from `dev` → `main`
4. **Reusable rule workflows** — replace the 33 per-repo copies of `package-rule-rc.yml` / `package-rule.yml` with a single `workflow_call` reusable workflow + thin per-repo caller stubs
---
## Background
### Current state problems
#### `publish.yml`
The current `publish.yml` has several critical problems that go beyond the stale action versions noted in #65:
| Problem | Detail |
|---|---|
| `checkout@v3`, `setup-node@v3`, Node 16 | Actions and runtime EOL |
| `push: dev` trigger | Publishes every push to `dev` — not developer-controlled |
| Auto-versioning via commit message parsing | Fragile; parses `[major]`/`[minor]`/`[patch]` tokens from commit messages |
| Version-bump PR automation | Creates an additional PR for every publish — noise and merge conflicts |
| `cat .npmrc` step | **Security bug** — leaks npm auth token to workflow run logs |
| `GITHUB_TOKEN: ${{ secrets.GH_TOKEN_LIB }}` at job level | Shadows the built-in `GITHUB_TOKEN`; token visible to all steps unnecessarily |
#### No version gate on `main`
Nothing currently prevents a developer from merging a PR to `main` while `package.json` still contains `-rc.N`. Without a gate, the `push: main` trigger in the new `publish.yml` would publish a prerelease version as `latest`.
#### `package-rule-rc.yml` / `package-rule.yml` — 35 per-repo copies
`frmscoe` has 33 rule repos that each carry their own full copy of these two workflow files. Combined with the 2 tazama-lf rule repos, there are **70 nearly-identical files**. Any bug fix or improvement must be applied 70 times. Several known bugs remain unaddressed (see Bug Inventory below).
---
## Library Dependency Tiers
All 33 `frmscoe` rule repos depend only on `@tazama-lf/frms-coe-lib` (plus `@tazama-lf/frms-coe-startup-lib` in rule-executer). They have no cross-rule dependencies. The full library tier ordering is:
```
TIER 1 — Foundations (release first, no internal deps):
@tazama-lf/frms-coe-lib — canonical source of all rule business logic types
@tazama-lf/auth-lib — authentication primitives
@tazama-lf/audit-lib — audit logging
TIER 2 — Rules and supplementary libraries:
@frmscoe/rule-001 .. rule-091 — all depend solely on frms-coe-lib@Tier1
@tazama-lf/frms-coe-startup-lib → frms-coe-lib
@tazama-lf/auth-lib-provider-keycloak → auth-lib ⚠️ RANGE (see note below)
TIER 3 — Services (consume Tier 1+2, no npm publish):
rule-executer, relay-service, event-director, typology-processor, etc.
```
### Special case: `auth-lib-provider-keycloak` range dependency
The dep is `"auth-lib": "^4.0.0-rc.4"`. npm behaviour: prerelease ranges only match prerelease versions on the **same** base. When `auth-lib@4.0.0` is published stable, this range stops resolving. Release-train must convert `^4.0.0-rc.4` → `^4.0.0`.
---
## Design Decisions
### Version string is the only publish signal
```bash
VERSION=$(jq -r '.version' package.json)
if [[ "$VERSION" == *-* ]]; then
npm publish --tag rc # X.Y.Z-rc.N → rc dist-tag
else
npm publish # X.Y.Z → latest dist-tag
fi
```
No event-name detection. The developer strips `-rc.N` manually in the release PR. `version-check.yml` enforces this gate.
### Triggers (replaces `push: dev`)
```yaml
on:
push:
branches: [main] # auto-publish stable on PR merge
workflow_dispatch: # developer manually publishes RC from dev
```
### Release-train: automated dep resolution
The release-train `workflow_dispatch` input is the target stable version (e.g. `4.0.0`). The workflow:
1. Validates input does not contain `-`
2. For each `@frmscoe`/`@tazama-lf` dep in `package.json`: queries `npm view @pkg dist-tags.latest` — fails if stable not yet published
3. Converts `X.Y.Z-rc.N` pinned deps → `X.Y.Z`
4. Converts `^X.Y.Z-rc.N` range deps → `^X.Y.Z`
5. Commits via GitHub API (Verified commit, satisfies branch protection)
6. Opens PR → `main` with configured reviewers
### Reusable workflow architecture
Each rule repo's stub is ~15 lines:
```yaml
# .github/workflows/package-rule-rc.yml (stub in each frmscoe rule repo)
on:
push:
branches: [dev]
workflow_dispatch:
jobs:
build:
uses: frmscoe/workflows/.github/workflows/package-rule-rc.yml@dev
with:
rule_number: "001"
rule_org: "frmscoe"
secrets: inherit
```
The canonical reusable file defined here holds the full job definition. `rule_org` input drives Dockerfile `sed` substitution (`@tazama-lf → @frmscoe`, `rule-901 → rule-NNN`).
---
## Bug Inventory (current `package-rule-*.yml` copies)
| # | Severity | Issue | Fix |
|---|---|---|---|
| 1 | 🔴 Critical | Docker tag hardcoded as `3.0.0` | Derive from `package.json` version |
| 2 | 🔴 Critical | `rule-002` only: single-quoted `sed` — shell variables never expand | Use double-quote pattern (already correct in all other rule repos) |
| 3 | 🔴 Critical | Version fetched via GitHub API without `?ref=` — reads wrong branch | Read from checked-out `package.json` directly |
| 4 | 🟠 Major | `cat .npmrc` step leaks auth token to workflow logs | Remove `cat` steps |
| 5 | 🟠 Major | `GITHUB_TOKEN` overridden at job level — shadows built-in | Remove; use `NODE_AUTH_TOKEN`/`GH_TOKEN` explicitly per step |
| 6 | 🟡 Minor | `npm install` used after deleting `package-lock.json` — non-reproducible | Use `npm ci` with lock file intact |
| 7 | 🟡 Minor | Action tags not SHA-pinned | Pin to verified SHAs |
| 8 | 🟡 Minor | No build provenance attestation | Add `actions/attest-build-provenance` |
| 9 | Architecture | 33 full copies in frmscoe alone — any bug fix requires 33 PRs | `workflow_call` reusable + caller stubs |
Bugs 4 and 5 are also present in `publish.yml` and will be fixed as part of the `publish.yml` replacement.
---
## frmscoe Rule Repo Inventory
All 33 repos currently have both `package-rule-rc.yml` and `package-rule.yml`. All will receive the thin caller stubs:
`rule-001`, `rule-002`, `rule-003`, `rule-004`, `rule-005`, `rule-006`, `rule-007`, `rule-008`, `rule-009`, `rule-010`, `rule-011`, `rule-012`, `rule-013`, `rule-014`, `rule-016`, `rule-017`, `rule-018`, `rule-019`, `rule-020`, `rule-021`, `rule-022`, `rule-024`, `rule-025`, `rule-026`, `rule-028`, `rule-030`, `rule-044`, `rule-045`, `rule-048`, `rule-054`, `rule-063`, `rule-074`, `rule-075`, `rule-076`, `rule-078`, `rule-083`, `rule-084`, `rule-090`, `rule-091`
(Numbers in the sequence that are absent — e.g. 015, 023 — have been confirmed non-existent in the org.)
---
## Implementation Plan
### Phase 1 — `publish.yml` replacement (resolves #65 item 1)
- [ ] **Replace `publish.yml`** with new canonical matching tazama-lf form:
- `@frmscoe` scope, Node 20, `actions/checkout@v4`, `actions/setup-node@v4`
- Triggers: `push: branches: [main]` + `workflow_dispatch`
- Version-string detection for dist-tag (`*-*` → `--tag rc`, else `latest`)
- Remove: auto-versioning, version-bump PR, `cat .npmrc`, job-level `GITHUB_TOKEN` override
- Add: explicit `NODE_AUTH_TOKEN` per step
### Phase 2 — Version enforcement (standalone)
- [ ] **Create `version-check.yml`**
- Trigger: `pull_request` targeting `main`
- Step: `jq -r '.version' package.json` → fail with clear message if result contains `-`
- Sync to: all rule repos via existing sync mechanism
### Phase 3 — Reusable rule workflows (resolves #65 item 2)
- [ ] **Create canonical `package-rule-rc.yml`** as `workflow_call` reusable in `frmscoe/workflows`
- Inputs: `rule_number` (string), `rule_org` (string, default `frmscoe`)
- Secrets: `GH_TOKEN_LIB`, `DOCKER_USERNAME`, `DOCKER_PASSWORD`, `SLACK_WEBHOOK_URL`
- Version derived from checked-out `package.json` (no API call, no `?ref=` needed)
- `npm ci`, no lock file deletion, no `cat .npmrc`, no job-level `GITHUB_TOKEN` shadow
- Docker tag: `$VERSION-rc` and `:rc` (moving pointer)
- `rule_org` input drives sed: when `frmscoe`, substitute `@tazama-lf → @frmscoe` and `rule-901 → rule-$rule_number` in Dockerfile
- [ ] **Create canonical `package-rule.yml`** as `workflow_call` reusable
- Same fixes; Docker tags: `$VERSION` and `:latest`
- [ ] **Update `sync-workflows.yml`** to stamp caller stubs (15 lines each) into all 33 frmscoe rule repos
- Stub-stamping logic derives `rule_number` from repo name (e.g. `rule-044` → `"044"`)
- Remove full-copy distribution of `package-rule-rc.yml` / `package-rule.yml` from sync
### Phase 4 — Release-train workflow
- [ ] **Create `release-train.yml`** in `frmscoe/workflows`
- Mirror of tazama-lf version with `@frmscoe` scope
- dep-resolution logic handles `@tazama-lf` tier-1/2 deps (rules depend on them)
- Commit via GitHub API (Verified), PR → `main`, optional back-bump to `dev`
### Phase 5 — Sync and distribution
- [ ] Include `version-check.yml` and `release-train.yml` in sync targets
- [ ] Run full sync to all 33 rule repos
- [ ] Verify: caller stubs correct, no full copies left in rule repos
---
## Security Notes
The existing `publish.yml` has a confirmed token-exposure bug:
```yaml
- name: Print .npmrc # ← THIS LEAKS THE TOKEN
run: cat .npmrc
```
This step prints the full contents of `.npmrc` which includes the `//npm.pkg.github.com/:_authToken=` line. While GitHub Actions masks known secret values in logs, this is a bad practice and will be removed in the replacement. The same applies to any similar `cat` step found in `package-rule-rc.yml` copies.
---
## Out of Scope
- `case-management-system`, `connection-studio`, `rule-studio` — under active development; dev teams adopt the new pattern once established
- Jenkins / deployment scripts
---
## Related Issues
- #65 — canonical workflow improvements (this issue extends and supersedes the `publish.yml` and `package-rule` items there)
- #63 — Node 16 removal from `node.js.yml` (parallel, independent track)
- [tazama-lf/workflows#30](https://github.com/tazama-lf/workflows/issues/30) — equivalent work in tazama-lf org (lead issue)
- [tazama-lf/workflows#27](https://github.com/tazama-lf/workflows/issues/27) — `publish.yml` `push: main` trigger
Contributor guide
Assessment
This issue has not been assessed yet.