picatz / picatz/flowstate

examples: "an example is something to paste and watch work" is false for about twenty of them, and the corpus already has both answers

Open
#2,001 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

Observed behavior

examples/README.md states the rule plainly: "Every other example runs as written, with no arguments, which is the rule: an example is something to paste and watch work." About twenty examples cannot. They name hosts under example.com, which resolves nowhere, so a local run stops at the first such step.

#1990 made that honest — the README now splits the network-marked examples by whether they reach a live service, a test holds the split, and the files that documented a local run say what it actually does. What it deliberately did not do is decide whether those files should stay that way, and the corpus currently answers the question both ways with no rule saying which applies:

  • Fictional host, read-and-adapt: enterprise-fund-transfer (ledger.internal.example.com), enterprise-customer-onboarding (provisioning.internal.example.com), deployment-reconciler, agentic-loop, the secret backends, crossing-dependencies, callback-address.
  • httpbin.org as a stand-in for a business service, genuinely runnable: order-fulfillment (whose README says url: stays a literal httpbin.org/status/200 regardless of the input), saga-provisioning, data-enrichment, fan-out-calls, progressive-rollout, loop-poll-until.

Both are defensible. order-fulfillment reserves stock and charges a card against httpbin.org and is none the worse for it; deployment-reconciler cannot use httpbin.org because its control plane has to answer differently per pass, and its README says so.

Two files sit awkwardly between the two, which is what surfaced this:

  • callback-address — its header narrated two local runs in detail ("Nobody answers, so the wait lapses"; "Watch what the register step below sends"), and the run dies at register before reaching any of it. #1990 replaced that with a pointer to flow test. Its register step only needs an endpoint that accepts a POST, which httpbin.org/anything does — and echoes, which would make "watch what it sends" literally true.
  • crossing-dependencies — five POSTs across three fictional hosts, and its subject (async: overlap) is the one thing no transcript can show anyway, so a real run buys less here.

What each file says about running, and the caveat it does not carry

Four files assert a host "resolves nowhere" without the resolver caveat examples/README.md and documentationOnlyHost carry after #1990 (raised by Codex on that PR, r3998604867, and dispositioned there as advisory rather than fixed):

File Line
examples/callback-address/workflow.yaml 19
examples/crossing-dependencies/workflow.yaml 34
examples/keychain-secret/README.md 27
examples/onepassword-secret/README.md 28

The claim is true on any resolver that does not synthesize records for a domain it does not own, and the corpus-wide caveat is stated once in the README where the rule is explained. Whether each example should repeat it — or say "nothing publishes a record for" instead — is the same decision as the one below, and belongs with it rather than as a separate sweep.

Desired outcome

One rule, written where an example author reads it, that decides which kind a new example is — and the existing corpus made consistent with it.

The rule probably is not "always one or the other". A plausible shape: an example whose subject is orchestration (compensation, fan-out, retries, gates) is runnable against httpbin.org, because what it teaches survives the endpoint being a stand-in; an example whose subject is the integration itself (a ledger's idempotency key, a control plane's convergence, a credential backend) keeps a named fictional host, because a generic echo would undercut the thing being shown, and says so where it says how to run it.

Acceptance criteria

  • examples/README.md's house style states the rule, and the "paste and watch work" sentence is either true of every example or scoped to the kind it is true of.
  • Each example that keeps a fictional host says so where it documents a run — already true at c6d2b07, and worth keeping checked rather than reviewed.
  • The four files above say the same thing about resolution that the README does, whatever that turns out to be.
  • callback-address and crossing-dependencies are decided explicitly under the rule rather than left as the two that prompted it.
  • No example that the rule makes runnable is left pointed at a host that does not resolve.

Constraints and dependencies

  • cmd/flow's TestExamplesREADMENetworkClaims (added in #1990) already holds the host set and the Network column's no; whatever rule lands should be expressible against it rather than beside it.
  • liveExampleHosts in that test is a standing permission for a real outbound request from a file people are invited to paste and run. Moving examples to httpbin.org widens what CI and a reader's machine reach; that is the cost side of the rule.
  • #1865 is the measured newcomer journey. This is one input to it — a reader who picks the most production-shaped example and runs it is exactly the journey that currently fails — but the rule is worth settling independently.

Open questions

  • Whether a third kind is needed for the examples whose endpoint must behave specifically (deployment-reconciler's per-pass control plane), which neither httpbin.org nor a fictional host serves, and which are exercised only by flow test.
  • Whether the conformance stand-in in pkg/flowstate/v1/internal/conformance/examples.go should be what runnable examples point at, rather than httpbin.org — it already models /anything, /status/{code} and the fictional internal domains, and would remove the corpus's one live third-party dependency.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XNNXyhgbbQqMhAdZWEEZ3C

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/README.md and cmd/flow's TestExamplesREADMENetworkClaims, then compare the listed workflow.yaml and README.md files with the existing example categories. The work is done when one documented rule settles the ambiguous examples, the four resolution claims agree with it, and the test can express the resulting host set and network expectations.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
documentation, testing-qa
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.