picatz / picatz/flowstate

flowtest: five things the loader knows and reports late or never — an empty `expect:`, a wrong `workflow:` path, a signal scripted after the gate's timeout, a type mismatch printed as quoting, and an unknown field with no suggestion

Open
#1,669 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug testing
Dominant language
Go
Stars
9
Forks
0
Avg merge
3h 3m
Merged PRs (30d)
509

Description

Observed behavior

Each reproduced at eb8172f.

  1. expect: {} passes. A case with no claim at all is green, and --fail-on-warning does not notice. The format's own principle (others: skipped exists so "adding a step fails loudly") does not apply to a case that asserts nothing.
  2. A wrong workflow: path is a run-time case failure. workflow: ./no-such-workflow.yaml reports loading workflow ... no such file or directory as the case's failure, with no position and no did-you-mean, while every other name in the format (ran:, signals, steps) is refused at load with a line.
  3. A signal scripted after the gate lapses gets no dedicated diagnostic. signals: [{name: deploy-approved, at: 48h}] against a gate with timeout: 24h produces three downstream failures; the transcript's t=24h0m5s approval waiting is the only clue. The loader holds both numbers.
  4. A type mismatch reads as a quoting difference. expect.outputs: {iterations: "1"} against an int fails with expected "1", got 1. The quotes are the whole message.
  5. An unknown field gets no suggestion. flow test on a file with outputs: where response: was meant answers unknown field "outputs" (the key is response, file.go:784; outputs is real only under expect:, file.go:998). run.go:1572,1601,1740 already produce did you mean %q? for step and signal names; field names, the first thing a newcomer gets wrong, get the bare refusal.

Desired outcome

Each is a load-time refusal or a load-time warning with a position:

  • an expect: with no claim is refused unless the case says expect: {nothing: true} or the spelling the format prefers for "this case only proves the run completes";
  • a workflow: that does not resolve is refused at load, positioned, with the nearest existing file suggested;
  • a signal whose at: is past every gate it could answer is refused at load naming the gate and its timeout;
  • output and input mismatches print expected string "1", got int 1 using the D1 type spelling;
  • unknown keys get the same did-you-mean the names get, over the keys legal at that position.

Acceptance criteria

  • Five fixtures under flowtest/testdata with # want lines (#1602's shape) pin each refusal.
  • examples/ and flow init's scaffold pass unchanged.

Constraints and dependencies

  • Adjacent: #1441, #1616, #1602, #1647 (type spelling in diagnostics).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the loader and diagnostic paths referenced at file.go:784 and 998, then inspect the did-you-mean handling in run.go:1572, 1601, and 1740. Add five fixtures under flowtest/testdata following #1602's # want shape, covering the listed load-time refusals or warnings. Run the flowtest fixtures and verify examples/ and flow init's scaffold remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.