connector/processor new: <30-min CI matrix, nightly drift job, upstream Windows-matrix PRs
- Dominant language
- Go
- Stars
- 610
- Forks
- 63
- Avg merge
- 12h 28m
- Merged PRs (30d)
- 57
Description
Follow-up to the \`conduit connector new\` / \`conduit processor new\` PR (roadmap: Phase 1 §"Plugin scaffolding", design doc \`docs/design-documents/20260707-connector-processor-scaffolding.md\`).
That PR ships the working commands and the \`pkg/scaffold\` engine (embedded template snapshot, Go-ported rewrite, per-language install+generate, verified \`go build ./...\`, git init). Three pieces from the design doc's acceptance criteria are explicitly deferred to this follow-up, per the design doc's own risk-tiering ("Windows-CI-green (AC-9) is an EXTERNAL dependency... tracked separately — not this repo's own deliverable"):
1. **AC-10: the `<30-min`, CI-measured matrix job** — ubuntu/macos/windows × connector/processor, on a runner with no preinstalled dev tooling, timing `new → make test → make build → smoke-run in a pipeline`; assert `< 1800s` (hard) + `< 600s` (warning, to catch creep). Publish duration per OS.
2. **Nightly drift job** — regenerate the embedded snapshot (`pkg/scaffold/template/{connector,processor}`) from upstream `conduit-connector-template`/`conduit-processor-template` `main` + latest SDK, build/test, fail loudly on drift before users hit it. Today the snapshot is a hand-synced pin (see `pkg/scaffold/template/embed.go`'s `ConnectorRef`/`ProcessorRef`).
3. **Upstream Windows-matrix PRs** — both `ConduitIO/conduit-connector-template` and `ConduitIO/conduit-processor-template`'s `test.yml` are `ubuntu-latest` only today. AC-9 ("generated `test.yml` green on `windows-latest`") needs a `windows-latest` unit-test leg added to each template's own CI — coordinated PRs to those two repos, not an in-repo edit here (an in-repo edit would diverge the vendored snapshot from upstream).
None of these block the command working today: `go build ./...` on a scaffolded repo is verified in-process as part of `scaffold.Generate` itself (see `pkg/scaffold/steps/build.go`), and the engine/commands are covered by unit + integration tests today.
Contributor guide
Research direction
Start with the design document and the existing scaffold path in pkg/scaffold/steps/build.go; inspect pkg/scaffold/template/embed.go for the pinned template references. Review the generated test.yml files in the upstream connector and processor templates before planning the CI and nightly jobs. Done means the timed matrix, drift detection, and coordinated Windows-matrix upstream changes meet the listed acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go
- Domain
- build-system, ci-cd
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100