galaxyproject / galaxyproject/foundry

author-galaxy-tool-wrapper: no validation step, no GalaxyUserTool schema, and the bundled prompt contradicts it

Open
#468 0 comments 0 reactions 0 assignees View on GitHub
agent/implement priority/mvp roadmap/substep
Dominant language
TypeScript
Stars
2
Forks
3
Avg merge
1d 22m
Merged PRs (30d)
63

Description

> **Posted by Claude (AI assistant) on jmchilton's behalf.** They did not author this text.

## Summary

`author-galaxy-tool-wrapper`'s sole output is a `GalaxyUserTool` YAML. The Mold:

- declares **`Validation: None`**;
- ships **no `GalaxyUserTool` / DynamicToolSources schema** in its cast bundle
(`references/schemas/` contains only `summary-nextflow.schema.json`);
- ships a prompt, `references/prompts/custom-tool-structured.md`, whose guidance
**contradicts** the real schema.

A UDT authored strictly to that prompt failed `gxwf validate-tool-source` with
**120 AJV errors**. The Mold has no way to detect this.

## The contradictions

| Bundled prompt teaches | `DynamicToolSources` requires |
| --- | --- |
| `format: fastq` (string) on inputs | `format: [fastq]` — an **array** |
| `default: 4` on numeric params | `value:` |
| `$(param_name)` shown first among reference forms | `$(inputs.param.path)` / `$(inputs.param)` |
| (silent) | `class: GalaxyUserTool` is **required** |
| (silent) | `container` is **required** |
| (silent) | `name` has `minLength: 5`; `id` must match `^[a-z][a-z0-9_-]*$` |

Output `format` is a plain string while input `format` is an array — an
asymmetry the prompt does not mention either.

## Reproducer

Author any UDT following `references/prompts/custom-tool-structured.md`'s
example verbatim, then:

```sh
gxwf validate-tool-source galaxy-user-tool.yml
# galaxy-user-tool.yml: 120 validation error(s)
# /inputs/0/format: must be array [type]
# /inputs/0/type: must be equal to constant [const]
# /class: must be equal to constant [const]
# ...
```

Fixing it required reading the schema out of
`@galaxy-tool-util/schema/dist/user-tool-source/schema.generated.js` on disk —
which the Mold's Runtime Notes explicitly forbid ("Do not read Foundry source
files at runtime; use only files packaged in this skill bundle").

## Proposed

Three mechanical changes:

1. Add a `Validation` entry to the Mold:
`gxwf validate-tool-source galaxy-user-tool.yml`. The tool exists, is already
a declared dependency elsewhere in the pipeline, and validates exactly this
artifact class.
2. Vendor the `GalaxyUserTool` schema into the bundle as a Load-Upfront
reference, the same way `summary-nextflow.schema.json` already is, so the
authoring step can consult the contract rather than a prose approximation.
3. Correct `custom-tool-structured.md` against that schema.

## Credit where due

The critic prompt's *"if you don't know the correct container image for a tool,
say so rather than guessing"* is load-bearing and worked. For egapx's NCBI C++
toolkit binaries the honest answer is a multi-GB `ncbi/egapx:0.5.2` image pulled
to run one binary — visibly a bad answer, correctly not replaced with a
plausible-looking BioContainer. The gap is only that the Mold cannot tell
whether its output is well-formed.

Found during `/test-pipeline nextflow-to-galaxy egapx`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the author-galaxy-tool-wrapper Mold and inspect its Validation configuration, then compare references/schemas/summary-nextflow.schema.json with references/prompts/custom-tool-structured.md. Use gxwf validate-tool-source galaxy-user-tool.yml and vendor the GalaxyUserTool schema as a Load-Upfront reference; done means the prompt matches the schema and the generated artifact validates without errors.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.