galaxyproject / galaxyproject/foundry

Prove the IWC maturation pipeline with runnable fixtures and a .ga entry case

Open
#531 0 comments 0 reactions 0 assignees View on GitHub
agent/implement enhancement priority/mvp roadmap/main
Dominant language
TypeScript
Stars
2
Forks
3
Avg merge
1d 22m
Merged PRs (30d)
63

Description

> **Posted by Claude (AI assistant) on behalf of @jmchilton.** Not authored by them personally.

Follow-up to #492, which is closed as design-complete. The Mold (#496, merged in #498), the four-phase `GALAXY WORKFLOW MATURATION` Pipeline (#508), the `iwc-maturation-report` contract, and five checklist fixtures all landed. What did not land is the evidence those specs assert against.

## The gap

`content/pipelines/galaxy-workflow-maturation/scenarios.md` states it plainly:

> They are checklist-shaped skeletons rather than runnable workflows: empty `steps`, empty `outputs`, and test jobs pointing at test data that does not exist. That is the right shape for judging a checklist pass, and it is enough for phase 3 — `gxwf validate` passes on all five. It is not enough for phase 4. No case below claims a green Planemo run, and none can be walked through phase 4 until test data is authored.

So the pipeline's fourth phase has never executed. #492's acceptance criterion — "the matured workflow is structurally validated **and actually run** through `run-workflow-test`" — is unverified, and with it acceptance scenarios 1, 2, 3, 6, and 7, each of which asserts a green or red test outcome.

The `.ga` entry path is unverified for the same reason. `content/molds/mature-galaxy-workflow-for-iwc/eval.md` declares a lossless-normalization property and then admits it is unbound:

> No committed fixture is `.ga` today, so this property is judged on a caller-supplied `.ga` entry rather than bound to a case.

## Scope

1. **Author one runnable fixture.** Real `steps`, real `outputs`, and test data small enough to commit, so `run-workflow-test` can actually execute. Extending `label-cleanup` or `safe-generalization` is likely cheaper than a sixth fixture — those two already carry the interface edits most likely to break a test, which is the boundary phase 4 exists to police.
2. **Demonstrate green.** Walk that fixture through all four phases and retain the Planemo evidence. This is acceptance scenarios 2 and 3.
3. **Demonstrate red, attributed.** Introduce an edit that breaks the test, confirm the run returns the failure with the phase-2 checklist items named as likely cause, and confirm nothing weakens an assertion to recover. This is acceptance scenario 6 and the eval properties "red evidence is retained and attributed across phases" and "no assertion is weakened to reach green".
4. **Add a `.ga` entry case.** A native `.ga` fixture that binds the lossless-normalization property to a case instead of to a hypothetical caller.
5. **Record a verification run.** `verification/pipelines/` currently holds only `minimal-nextflow`; the maturation pipeline should sit beside it.

## Out of scope

Everything #492 already deferred — fork/branch/IWC-Lab pull request, #491 against that PR, check monitoring, waiver policy, publication bundle and manifest, Dockstore and deployment canaries, automated remediation. Also out of scope: any change to the checklist semantics. If a real run shows the Mold is wrong about something, that is a separate issue against the Mold.

## Done when

- One fixture runs end to end through all four phases, green, with retained Planemo evidence.
- The same fixture, deliberately broken, runs red with the failure attributed to the phase-2 change and no assertion weakened.
- A `.ga` fixture binds the normalization property to a committed case.
- A verification run for the maturation pipeline is recorded.
- `mature-galaxy-workflow-for-iwc` and `galaxy-workflow-maturation` can move off `status: draft` on the strength of that evidence.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with content/pipelines/galaxy-workflow-maturation/scenarios.md and content/molds/mature-galaxy-workflow-for-iwc/eval.md, then inspect the label-cleanup and safe-generalization fixtures. Run the selected fixture through gxwf validate and run-workflow-test, retaining green and deliberately broken evidence with the failure attribution. Add a committed .ga case and record the maturation verification run under verification/pipelines/.

Written by the indexing model from the issue text.

Assessment

Domain
devops, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.