picatz / picatz/flowstate

plugins: five modules reinvent the shared substrate — roughly a thousand duplicated lines the sixth plugin will copy again

Open
#1,333 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design engine kind/design-record testing
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

Problem

The duplication across plugins/{codex,git,github,sql,vcs} is not incidental — the source names it, repeatedly, and always for the same reason: separate Go modules with no shared non-root package to put it in. The ledger, grep- and diff-verified:

Duplicate Where Approx. lines
packBoundedStorer git/packbound.go, vcs/packbound.go — both doc comments say "kept identical in shape" to the other ~125 ×2
bounded write stream vcs/diff.go boundedPatchWriter, codex/diff.go boundedWriter — comment: "copied in shape… a different module, so not imported" ~30 ×2
resolveSecret+envSegment+tokenFromValue git/secrets.go, vcs/secrets.go (identical modulo prefix), github/auth.go (same shape) ~95 ×3
isNetworkUnavailable git/errors.go, vcs/errors.go, github/errors.go — byte-identical ~8 ×3
validateRepositoryURL+validateRevision+clampMaxCommits git/validate.go, vcs/validate.go ~65 ×2
installEgressPolicy git/clone.go, vcs/clone.go, github/client.go (with a real unit inconsistency: seconds-typed vs duration-typed timeout constants) ~14 ×3
reachable/ test harness (buildPlugin/openHost/spoofing/schema-refusal assertions) all five plugins/*/reachable/reachable_test.go, 1,682 lines total, ~100-line shared tail each ~500

Every plugin module already requires and replaces the root module and already imports pkg/flowstate/v1/netpolicy and pkg/flowstate/v1/secrets across that boundary, so a shared home (a plugintoolkit package, and a test-side kit under pkg/flowstate/v1/plugin for the reachable/ harness) needs no new module wiring — it is subtraction behind calls that already cross this exact seam. This is the repository's own bar for a new abstraction (remove real duplication) met with measured numbers, and it is the change that makes plugin #6 cost less to build correctly: today each new plugin re-copies the harness to earn its reachability proof and re-derives the same egress/secret/validation/bounding conventions by citation.

Conventions propagated by citation are the softer half of the same problem: sql/bounds.go and codex/bounds.go exist "for the reason plugins/vcs/validate.go gives"; three of five plugins hold README example fences byte-identical to disk via readme_test.go, two do not; sql/errors.go's own doc comment records two classification bugs of exactly the kind a shared, tested classifier pattern would have caught once.

Desired outcome

A design record deciding the extraction: which rows above land in a shared package (the test harness is the safest first slice — test-only, mechanical, largest single win), which stay per-plugin for real reasons (per-backend error taxonomies argue for a parameterized helper, not a replacement; the five test doubles are correctly not shared), and the API shapes. Explicitly not a manifest maturity/tier field — #239 settled that a self-declared danger class is a value a malicious plugin sets.

Acceptance

  • The extraction lands with no behavior change: make test-plugins green, all five reachable/ suites keep their assertions (pre-registration refusal, schema refusal, spoofing proof) through the shared kit.
  • The next new plugin's reachable_test.go is ~30 lines of calls, not ~100+ of copies.
  • The unit inconsistency between the three installEgressPolicy copies is gone (or subsumed by #1332's grant, which should be written once here rather than five times).

Adjacency

#1330 (secrets migration removes the biggest production-code row), #1332 (egress contract owns that row's future), #713 (owns the external, language-neutral conformance harness — this is the in-tree half), #715 (ordered the outside-facing track; this is its internal complement).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin by reading the duplication ledger and the five plugins/*/reachable/reachable_test.go suites, then run make test-plugins to establish the current behavior. Define the shared production and test-side package boundaries and API shapes, preserving each suite’s pre-registration refusal, schema refusal, and spoofing assertions. Done means behavior is unchanged and a new reachable_test.go is about 30 lines.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
testing, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.