HarperFast / HarperFast/integration-testing

Shared GitHub Actions workflow for integration tests across Harper repos

Open
#25 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
17d 7h
Merged PRs (30d)
2

Description

## Idea

Most Harper repos that use the `integration-testing` framework have nearly identical `integration-tests` GitHub Actions workflows. Comparing five of them:

| Repo | Node default | Build | Fixtures | Shards | Bespoke extras |
|---|---|---|---|---|---|
| oauth | 22, 24 | ✅ | ✅ | — | none (pure simple tier) |
| generic plugin | 20/22/24 | optional | — | — | none |
| nextjs | 20, 22, 24 | ✅ | ✅ | — | playwright install, `harper_ref` override, trace artifacts |
| harper-pro | 22, 24, 26 | ✅ | — | 4 | submodules, re2 rebuild, legacy `harperdb@4`, concurrency cap |
| harper (core) | 22, 24, 26 | ✅ | — | 6 | Windows, Bun, uWS, legacy install |

The common skeleton — matrix generation → checkout → node setup → `npm ci` → build/fixtures → run `test:integration` → upload logs on failure — is identical across all of them. Everything that differs lives in the *middle steps*. That ~60% boilerplate is a good consolidation target; the bespoke middles are not.

## Proposed approach: two shared artifacts, hosted here, versioned with the framework

The divergence lives mid-job, and a reusable workflow is all-or-nothing at the job level (you can't inject a step into its middle). So a single mechanism isn't enough — pair two:

1. **A reusable workflow** (`.github/workflows/integration-tests.yml`, `on: workflow_call`) that owns the entire simple-tier pipeline (matrix gen, setup, run, logs-on-failure). Inputs: `node-version` / `all-node-versions`, `build`, `install-fixtures`, `test-command`, `timeout-minutes`. Callers shrink to ~8 lines. Covers oauth + any plain plugin.

2. **A composite action** (`.github/actions/setup/action.yml`) that owns just the shared setup steps (node + `npm ci` + optional build/fixtures), dropped into a caller's own job. For repos that must interleave custom steps — nextjs (playwright, `harper_ref`), harper-pro/core (sharding, native rebuilds). The action deliberately does *not* checkout (harper-pro needs `submodules: recursive`). Caveat: a composite action can't wrap a failure-triggered log upload around the caller's test step, so heavy-tier callers repeat those few lines.

### Example: oauth's entire workflow becomes

```yaml
name: Integration Tests
on:
push: { branches: [main] }
pull_request:
workflow_dispatch:
inputs:
node-version: { type: choice, default: all, options: [all, '22', '24'] }
jobs:
test:
uses: HarperFast/integration-testing/.github/workflows/integration-tests.yml@v1
with:
node-version: ${{ inputs.node-version || 'all' }}
all-node-versions: '[22, 24]'
build: npm run build
install-fixtures: npm run install:fixtures
timeout-minutes: 5
```

### Example: nextjs / harper-pro keep their own job, reuse only setup

```yaml
steps:
- uses: actions/checkout@v6.0.3
# with: { submodules: recursive } # harper-pro
- uses: HarperFast/integration-testing/.github/actions/setup@v1
with:
node-version: ${{ matrix.node-version }}
build: npm run build
# --- project-specific middle: playwright / harper_ref / re2 / shards ---
- run: npx playwright install --with-deps chromium
- run: npm run test:integration
env: { HARPER_INTEGRATION_TEST_LOG_DIR: /tmp/harper-test-logs, FORCE_COLOR: '1' }
```

## Where to draw the line

- **Use the reusable workflow** if your CI is just matrix → build → run → logs (oauth, most future plugins).
- **Use only the composite action** the moment you need a step in the middle (playwright, `harper_ref`, sharding, native rebuilds) — keep your job, dedupe the setup.
- **harper core stays bespoke** — its matrix (Windows × Bun × uWS × 6 shards × legacy install) is divergent enough that routing it through the shared workflow would bury it in conditionals. It can still open each job with the shared setup action.

## Open questions / trade-offs to discuss

- **Blast radius vs. drift.** Consolidation trades N maintained files for one shared surface — a bad change can break every consumer's CI at once. Mitigate by pinning callers to a tag/SHA (never `@main`) and versioning the workflow/action alongside the framework release, so CI config and framework move in lockstep. Is that coupling worth it vs. a copy-paste scaffold template (no runtime coupling, but drifts)?
- **Home.** This repo (co-located + versioned with the framework) vs. a shared `HarperFast/.github` repo.
- **Scope of inputs.** How much variance (playwright toggle? shard count? legacy install?) belongs in the reusable workflow before it's better left to the composite action?
- **Secrets.** Callers add `secrets: inherit` when tests need them — worth documenting as a convention.

If we pursue this, next step would be the two files + a short `docs/ci.md` adoption guide, then converting one simple consumer (oauth) as a proof of concept.

sent with Claude Opus 4.8

Contributor guide

Open the contributing guide

Research direction

Read the existing integration-tests GitHub Actions workflows in the oauth, generic plugin, nextjs, harper-pro, and harper repos first, comparing their shared and bespoke steps. Then assess the proposed .github/workflows/integration-tests.yml, .github/actions/setup/action.yml, and docs/ci.md; done means the two artifacts exist and oauth is converted as a proof of concept.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.