sdk: no author-facing test kit — testing a task means launching the binary through a host or hand-building `map[string]*Value`, and neither checks the outputs, the claims, or the classification the host will read
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 9
- Forks
- 0
- Avg merge
- 3h 3m
- Merged PRs (30d)
- 509
Description
Problem
The SDK's public surface is Main/Run, the options, DecodeInputs/EncodeOutputs/Literal, the egress and caller helpers, and the error constructors (pkg/flowstate/v1/plugin/sdk/*.go). There is no test package. An author who wants to test a task has two routes, and the first-party plugins use both:
- Call an internal function directly with hand-built values (
plugins/git/commit_push_test.gocallsdoCommitPush, bypassing the input gate on purpose, as its own comment says). This tests the author's Go and nothing about the contract: the descriptor bridge is never crossed, so #1456 (arepeated CommitoutputEncodeOutputsrefuses) shipped green. - Build the binary and launch it through a real host (
plugins/*/reachable/, 2,057 lines overinternal/pluginreachtest, per #1476). This proves reachability and assertsValidateSourceonly; it is in-tree tooling an external author cannot import.
Fixtures are the same story. Egress in a test means setting FLOWSTATE_EGRESS_POLICY_B64 by hand (sdk/egress_test.go); the caller identity and mode are installed by an unexported contextWithCaller; a resolved secret arrives as a plain value with no way to assert the input was declared to accept one. So the things the host enforces — outputs against output_message (#1476), the manifest's name lists against the descriptor (#1477), how a returned error classifies and whether it is retried (#184) — are exactly the things an author's test cannot see.
Desired outcome
A pkg/flowstate/v1/plugin/sdk/sdktest package, importable by any plugin module, that runs a task in-process the way the host would read it:
sdktest.Call(t, task, in proto.Message, opts...) (out proto.Message-typed, sdktest.Result)encodesinthrough the same value bridge the host uses, runs the manifest and descriptor checkssdk.Runruns, invokesFn, decodes the outputs against the declared output descriptor and fails on any undeclared or mis-kinded field (the author-side half of #1476), and returns the classification as the host computes it: kind, retryable, retry-after, unknown outcome, code.- Fixtures as options:
WithCaller(identity, namespace, mode),WithEgressPolicy(document)andWithDeploymentDefaultEgress(),WithSecret(input, value)(refusing an input not declared inSecretInputs, as the host would),WithScope(...),WithProgress(func(phase)). sdktest.Conform(t, plugin): the manifest/descriptor coherence checks (#1477), the error round-trip (#184 gap 1: constructor → wire → hostTaskError→ text, message exactly once), the relaunch-stability check (a manifest that varies per call), and a stdout-hygiene check, all in-process with no binary and no socket.
The in-tree reachable/ harness stays the launch-based proof (#1333 owns extracting it); #713 owns the black-box binary conformance. This is the layer beneath both, and the one an outside author gets from go get alone.
Acceptance criteria
plugins/slackand one other first-party plugin move their task tests ontosdktest.Callwith no loss of assertions, and #1456's shape is caught by the kit before the host is involved.sdktest.Conformfails on a plugin whoseSecretInputsnames a field its descriptor does not have, and on anFnthat returns an output field its descriptor does not declare.- The chapter-two plugin in PLUGINS.md gains a test file using the kit, and the page shows it.
- The kit has no dependency on
internal/packages, so it is usable from a module outside this repository (a CI leg proves it, the same one #1496 asks for).
Adjacent: #1333, #713, #1476, #1477, #184, #1341 (concurrency contract the kit can exercise by calling Fn from two goroutines).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pkg/flowstate/v1/plugin/sdk/*.go, especially sdk.Run and the existing egress tests, then compare the reachable/ harness and internal/pluginreachtest behavior. Trace the value bridge, manifest and descriptor checks, and error classification before planning sdktest.Call and sdktest.Conform. Done means the stated acceptance criteria pass for plugins/slack, another plugin, the chapter-two example, and an external module without internal/ dependencies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- developer-experience, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100