galaxyproject / galaxyproject/foundry

paper-to-cwl: a missing cwl-test-plan producer, and find-test-data used off its target axis

Open
#448 0 comments 0 reactions 0 assignees View on GitHub
priority/v2 roadmap/substep
Dominant language
TypeScript
Stars
2
Forks
3
Avg merge
1d 22m
Merged PRs (30d)
63

Description

> Filed by Claude (AI assistant) on jmchilton's behalf — they did not write this text.

`paper-to-cwl` carries two of the three remaining structural validator warnings. Both look like accepted design gaps from the warning text alone. Neither is.

```
content/pipelines/paper-to-cwl/index.md:
WARN phases[6]: no prior phase produces any input_artifact for role 'galaxy-interface'
(freeform-galaxy-interface, nextflow-galaxy-interface, cwl-galaxy-interface)
WARN phases[7]: input_artifact 'cwl-test-plan' has no prior phase producing it in this pipeline
```

## 1. `phases[7]` — the CWL target is missing a paper-sourced test-plan producer

Every `*-test-plan` producer and consumer in the inventory:

| producer | emits |
| --- | --- |
| `freeform-summary-to-galaxy-test-plan` | `galaxy-test-plan` |
| `nextflow-test-to-galaxy-test-plan` | `galaxy-test-plan` |
| `cwl-test-to-galaxy-test-plan` | `galaxy-test-plan` |
| `changeset-to-galaxy-test-plan` | `galaxy-test-plan` |
| `nextflow-test-to-cwl-test-plan` | `cwl-test-plan` |

Four producers for the Galaxy target, one for CWL — and the one that exists is the Nextflow-sourced one, because nf-test fixtures are there to translate. A paper has no test suite to convert, so on a paper-sourced run nothing produces `cwl-test-plan` and `implement-cwl-workflow-test` (its only consumer) has no input.

The sibling settles it. `paper-to-galaxy` has the same source and the same test-data branch, and it carries a test-plan phase between them:

```yaml
# paper-to-galaxy # paper-to-cwl
- branch: test-data-resolution - branch: test-data-resolution
chain: [paper-to-test-data, chain: [paper-to-test-data,
find-test-data, user-supplied] find-test-data, user-supplied]
- mold: freeform-summary-to-galaxy-test-plan # ← nothing here
- mold: implement-galaxy-workflow-test - mold: implement-cwl-workflow-test
```

`nextflow-to-cwl` also runs the test-plan phase before `implement-cwl-workflow-test`. `paper-to-cwl` is the only pipeline reaching that Mold without one, so this is an omission rather than a decision — and the missing Mold's name falls out of the symmetry: **`freeform-summary-to-cwl-test-plan`**.

`content/meta/harness-pipelines.md` §"PAPER → CWL" has the same skip in its narrative (7 → 8 goes branch → `implement-cwl-workflow-test`), so the note and the design doc agree with each other and disagree with every sibling. Both need the phase if it lands.

Worth deciding rather than assuming: the paper-sourced Galaxy producer takes the `freeform-summary` and produces a plan, so the CWL analogue plausibly does the same — but a paper yields accessions and figures, not fixtures with expected outputs, and whether that is a test *plan* or only test *data* is the actual question. The test-data branch already produces `test-data-refs`. If the answer is "only data", then the fix is at `implement-cwl-workflow-test`'s input surface, not a new Mold.

## 2. `phases[6]` — `find-test-data` is a Galaxy-target Mold on a CWL journey

`find-test-data` is tagged `target/galaxy`, and all three of its `galaxy-interface` role inputs are Galaxy interface briefs:

```yaml
- id: freeform-galaxy-interface # from freeform-summary-to-galaxy-interface
role: galaxy-interface
- id: nextflow-galaxy-interface # from nextflow-summary-to-galaxy-interface
role: galaxy-interface
- id: cwl-galaxy-interface # from cwl-summary-to-galaxy-interface
role: galaxy-interface
```

`paper-to-cwl`'s design phase emits `freeform-cwl-design`, which declares no `role` at all. So the role is not merely unsatisfied here — nothing on a CWL-target journey could satisfy it.

Of the six pipelines whose test-data branch names `find-test-data`, five target Galaxy. `paper-to-cwl` is the only CWL-target one, which is why it is the only place this surfaces. (Note the first link in the chain is fine: `paper-to-test-data` needs only `freeform-summary`, which `phases[0]` produces. Only the fallback is off-axis.)

Three ways out, and they are not equivalent:

1. Generalize the role — the Mold wants *the interface decisions that pin input shapes*, and `freeform-cwl-design` carries those for CWL. A rename (`interface-brief`?) plus adding `freeform-cwl-design` to the role would fix it without new Molds, but re-scopes a Mold currently tagged `target/galaxy`.
2. A CWL-specific sibling, mirroring how the interface briefs are already per-source-per-target.
3. Accept it: the fallback runs with less context on CWL journeys and the role is genuinely optional. If so the validator should be able to express that, because right now "accepted gap" and "unnoticed bug" render identically.

## Not doing yet

Leaving `paper-to-cwl`'s missing `eval.md` / `scenarios.md` warnings in place deliberately. There is no paper fixture vendored anywhere in the repo, so `scenarios.md` cases could not bind a real fixture today, and writing the companions before these two gaps are settled would silence a warning that is currently reporting something true: this pipeline has not been walked. The companions are worth writing after a first walk, not to clear a check.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with content/pipelines/paper-to-cwl/index.md and compare its test-data branch with paper-to-galaxy and nextflow-to-cwl. Read content/meta/harness-pipelines.md §"PAPER → CWL", then inspect the find-test-data and implement-cwl-workflow-test inputs. Done means the design decision is resolved, the pipeline and narrative agree, and the structural validator warnings are correctly addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
yaml
Domain
tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.