Testing + debugging: the remaining dozen, in order
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 9
- Forks
- 0
- Avg merge
- 3h 3m
- Merged PRs (30d)
- 509
Description
The slate after #1109, capped at twelve and ordered by what each buys. Every claim below is cited against the tree at 9040214c rather than remembered, per CLAUDE.md's rule about sketches.
Where it stands. The debugger has three fronts (a prompt on flow test --debug, the same on flow run local --debug, a scripted session over MCP as flowstate_debug), the test DSL has cases:/expect.check:/vars:/testdefaults.yaml, and the conformance rule is walked rather than imitated. What follows is what is missing, not what is broken.
The debugger (#928)
1. DAP adapter — editors drive the session themselves. Medium-large.
The core is protocol-agnostic and the mapping is unusually clean because the units are semantic: a breakpoint is a step id, a stack frame is the call: chain, a scope pane is inputs/vars/steps/loop bindings. editors/vscode is a real extension with no debuggers contribution, and docs/EDITORS.md:423 records Helix printing Configured debug adapter: None. This is the front that makes the debugger visible to people who never read a --help.
2. A real TUI for the prompt, with CEL completion. Medium.
Today's session is a themed prompt; the ask on #928 was a REPL "crafted to the same bar as the rest of the CLI" with high-quality completion. The completer already exists and is scope-aware — completeInExpression/bareCandidates over refScope (flowfile/lsp/completion.go) — and mirrors the validator's idea of what a name may be. Handing it a live scope instead of a static one is reuse, not a third spelling. A scope pane and a step list ride the same internal/ui theme the appearance job already pins.
3. flow debug replay <script>. Small.
The record half is built and spent nowhere: Session.Script() has exactly one non-test consumer, the MCP answer. A script in an issue should be a reproduction anyone can run — and it is also the natural fixture shape for an examples-corpus debugging case, which is what "a capability is not done until an example exercises it in CI" would want here.
4. Durable debug (slice 2). Large, schema-first.
The decided design: the file declares (debug: beside signals:), deployment narrows and never widens, no stanza = not debuggable; an abandoned session's lease expires and resumes the run, with pause/expiry/auto-resume all attributable. Wire messages sequence after #923. This is the one that needs the most care and the least hurry.
The test DSL (#1072)
5. Computed vars:. Medium. Design pass posted on #1072 and awaiting ack: the fence decides (a fenced value evaluates, an unfenced one stays literal — strictly additive, since every fenced value is refused today), sibling reads land via dependency sort with a cycle diagnostic (the branch docs/DSL.md left open on purpose: "allowing it later is additive"), evaluation is per-case, and a computed var may read vars. siblings and nothing else.
6. Case- and row-level vars:. Small-medium, rides 5. Merged defaults→case→row, explicit beats inherited, like every sibling list. A computed entry may read an inherited one; the sort runs over the merged set.
The editor (#1110)
7. Loader-owned positions (slice 2). Medium.
Today a semantic refusal from the loader carries prose and no position, so it anchors by heuristic — at the named test's name: line when the message names exactly one, at the top otherwise (flowfile/lsp/testfile.go). Slice 2 deletes that heuristic rather than growing it, and gives a broken testdefaults.yaml its own URI instead of the suite's document start (which is where #1109 parked it as the honest interim).
8. The test language in the editor, beyond diagnostics. Medium.
speaksFlowfile() is kind == docWorkflow (flowfile/lsp/store.go:52), and completion, hover, code actions, formatting and symbols all gate on it. So a *.test.yaml gets diagnostics and nothing else: no completion for expect. keys, for the task names a stubs: entry may name, or for the case names --run will match. The test language is a language; it should behave like one.
The harness
9. make coverage in CI. Small. CLAUDE.md names it as a follow-up and nothing under .github/ runs it. It is a map rather than a gate — no percentage enforced — but a map nobody draws is one nobody reads.
10. A repeatable vacuity check. Medium. This session caught four tests that asserted nothing by hand: computed indices that never reached the cap they claimed to test, absence assertions that would pass on empty output. Four hand-catches is not a mechanism; the fuzztargets/conformance-callers mold (walk the tree, fail on disagreement) is the shape that would be.
11. flowstate_debug over a real local run. Small-medium. Deliberately not in #1109: the bare-run path inherits run_local's egress posture and the reveal question that flow run local --debug answers with a refusal. Worth doing once, carefully, rather than by extending the stubbed tool's arguments.
12. Both-drivers conformance for the debugger seam. Small. v1.Debugger is asked at each step boundary on the local driver. The durable driver has no counterpart yet and will not until slice 2 — but the conformance package is where that agreement gets stated, and stating it now (even as a case that documents the asymmetry) is cheaper than discovering it in slice 2.
Nothing here is blocked on anything but #1109 merging. Sizes are honest guesses, and 1, 2 and 5 are where I would spend the next day.
Generated by Claude Code
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
This is a twelve-item roadmap rather than one self-contained change. After #1109 merges, choose a scoped item and start with its cited entry points, such as flowfile/lsp/completion.go, flowfile/lsp/testfile.go, flowfile/lsp/store.go, .github/, or the conformance package. Done means the selected capability has implementation coverage and the relevant tests, examples, or CI checks exercise it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- devtools, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100