plugins: five modules reinvent the shared substrate — roughly a thousand duplicated lines the sixth plugin will copy again
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 9
- Forks
- 0
- Avg merge
- 3h 3m
- Merged PRs (30d)
- 509
Description
Problem
The duplication across plugins/{codex,git,github,sql,vcs} is not incidental — the source names it, repeatedly, and always for the same reason: separate Go modules with no shared non-root package to put it in. The ledger, grep- and diff-verified:
| Duplicate | Where | Approx. lines |
|---|---|---|
packBoundedStorer |
git/packbound.go, vcs/packbound.go — both doc comments say "kept identical in shape" to the other |
~125 ×2 |
| bounded write stream | vcs/diff.go boundedPatchWriter, codex/diff.go boundedWriter — comment: "copied in shape… a different module, so not imported" |
~30 ×2 |
resolveSecret+envSegment+tokenFromValue |
git/secrets.go, vcs/secrets.go (identical modulo prefix), github/auth.go (same shape) |
~95 ×3 |
isNetworkUnavailable |
git/errors.go, vcs/errors.go, github/errors.go — byte-identical |
~8 ×3 |
validateRepositoryURL+validateRevision+clampMaxCommits |
git/validate.go, vcs/validate.go |
~65 ×2 |
installEgressPolicy |
git/clone.go, vcs/clone.go, github/client.go (with a real unit inconsistency: seconds-typed vs duration-typed timeout constants) |
~14 ×3 |
reachable/ test harness (buildPlugin/openHost/spoofing/schema-refusal assertions) |
all five plugins/*/reachable/reachable_test.go, 1,682 lines total, ~100-line shared tail each |
~500 |
Every plugin module already requires and replaces the root module and already imports pkg/flowstate/v1/netpolicy and pkg/flowstate/v1/secrets across that boundary, so a shared home (a plugintoolkit package, and a test-side kit under pkg/flowstate/v1/plugin for the reachable/ harness) needs no new module wiring — it is subtraction behind calls that already cross this exact seam. This is the repository's own bar for a new abstraction (remove real duplication) met with measured numbers, and it is the change that makes plugin #6 cost less to build correctly: today each new plugin re-copies the harness to earn its reachability proof and re-derives the same egress/secret/validation/bounding conventions by citation.
Conventions propagated by citation are the softer half of the same problem: sql/bounds.go and codex/bounds.go exist "for the reason plugins/vcs/validate.go gives"; three of five plugins hold README example fences byte-identical to disk via readme_test.go, two do not; sql/errors.go's own doc comment records two classification bugs of exactly the kind a shared, tested classifier pattern would have caught once.
Desired outcome
A design record deciding the extraction: which rows above land in a shared package (the test harness is the safest first slice — test-only, mechanical, largest single win), which stay per-plugin for real reasons (per-backend error taxonomies argue for a parameterized helper, not a replacement; the five test doubles are correctly not shared), and the API shapes. Explicitly not a manifest maturity/tier field — #239 settled that a self-declared danger class is a value a malicious plugin sets.
Acceptance
- The extraction lands with no behavior change:
make test-pluginsgreen, all fivereachable/suites keep their assertions (pre-registration refusal, schema refusal, spoofing proof) through the shared kit. - The next new plugin's
reachable_test.gois ~30 lines of calls, not ~100+ of copies. - The unit inconsistency between the three
installEgressPolicycopies is gone (or subsumed by #1332's grant, which should be written once here rather than five times).
Adjacency
#1330 (secrets migration removes the biggest production-code row), #1332 (egress contract owns that row's future), #713 (owns the external, language-neutral conformance harness — this is the in-tree half), #715 (ordered the outside-facing track; this is its internal complement).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin by reading the duplication ledger and the five plugins/*/reachable/reachable_test.go suites, then run make test-plugins to establish the current behavior. Define the shared production and test-side package boundaries and API shapes, preserving each suite’s pre-registration refusal, schema refusal, and spoofing assertions. Done means behavior is unchanged and a new reachable_test.go is about 30 lines.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100