galaxyproject / galaxyproject/foundry

research: flesh out nextflow-to-galaxy-reference-data-mapping

Open
#219 0 comments 0 reactions 0 assignees View on GitHub
priority/mvp roadmap/off
Dominant language
TypeScript
Stars
2
Forks
3
Avg merge
1d 22m
Merged PRs (30d)
63

Description

## Summary

Flesh out the stub research note `content/research/nextflow-to-galaxy-reference-data-mapping.md`. The stub captures the v1 design posture; the corpus and detailed guidance are thin and need work before downstream Molds (`nextflow-summary-to-galaxy-interface`, `nextflow-summary-to-galaxy-data-flow`, future `paper-summary-to-galaxy-design`) can lean on it without re-deriving rules per pipeline.

Surfaced from the sarek emulation (2026-05-08); see refinement journal at `content/molds/nextflow-summary-to-galaxy-interface/refinements/2026-05-08-sarek-emulation.md`.

## v1 posture (already pinned in the stub)

- Translate every nf-core reference asset to an **explicit optional Galaxy `data` input**.
- When a derived input (index, dict, tabix) is unset, the Galaxy tool wrapper rebuilds it from the FASTA inside the wrapper — matching what existing IWC wrappers (BWA, BWA-MEM2, GATK4, Picard) already do.
- Stay off `dbkey` / cached data tables in v1. Galaxy folks who care about portable reference UX increasingly avoid them; cross-instance reproducibility favors explicit inputs.
- iGenomes the *key* (`params.genome = 'GATK.GRCh38'`) does not survive translation. The Galaxy workflow surface starts at the resolved per-asset paths, not the key.

This posture is the part that needs the least change. The rest of the note is what needs work.

## Concrete TODOs (each ~one PR)

1. **Audit the iGenomes attribute set.** The stub lists ~12 derived params (fasta, fai, dict, bwa, bwamem2, dragmap, dbsnp, dbsnp_tbi, known_indels, known_indels_tbi, germline_resource, germline_resource_tbi, intervals, pon). Confirm by reading `nf-core/sarek/conf/igenomes.config` end-to-end and `nf-core/configs` for the central iGenomes profile. Document which attributes appear across pipelines vs which are sarek-specific.

2. **Per-asset translation table.** The stub has a sketch table for sarek. Generalize: which assets always become Galaxy inputs, which become tool-internal rebuilds, which depend on workflow scope. Cross-check against rnaseq (lighter — no BQSR, but salmon/STAR indexes), bacass (lighter — only FASTA), taxprofiler (different — krakendb, centrifugedb).

3. **Compute-if-missing source detection.** Add concrete grep/regex patterns for the source `if (!params.fasta_fai) { ... }` shape. Ideally extract automation guidance the interface Mold can apply: "for each reference param, scan main.nf + subworkflows for a `!params.` guard around an index-building process; if present, mark the Galaxy input optional."

4. **Heavy-index threshold.** The stub mentions ~3 reuses as the cutoff between in-tool rebuild vs explicit workflow-tier build step. This is a guess. Validate against existing IWC variant-calling and rnaseq exemplars — does anyone build an index as a workflow step? At what reuse count?

5. **Test fixtures.** nf-core test profiles ship a tiny FASTA + index bundle (typically `nf-core/test-datasets`). IWC test fixtures do similar. Document the mapping; this fragment gets pulled into `nextflow-test-to-galaxy-test-plan` later.

6. **PoN, germline-resource, gnomAD.** Big per-build VCFs the user must supply (no in-tool rebuild possible). Flag explicitly so the interface Mold marks them as required-when-needed rather than optional. Add a subsection.

7. **`dbkey` annotation propagation.** Even with the no-cached-tables posture, `dbkey` may show up if a user annotates inputs. Document what the data-flow Mold should do (preserve through map-overs and `Apply rule` steps).

8. **Cross-check against IWC exemplars.** Run the recommended posture against `variant-calling/generic-variant-calling-wgs-pe`. Does the exemplar's reference handling match the v1 posture, or does it deviate? If deviation, document why and pick.

9. **Galaxy reviewer review.** The "Galaxy folks who care don't like dbkeys" claim in the stub is paraphrased from a single conversation. Worth verifying with at least one IWC reviewer or a Galaxy training community thread before promoting out of `status: draft`.

10. **Promote out of draft.** Once 1-9 are done, change `status: draft` → `status: live`, and update the related Mold pages to wiki-link this note in their `references[]`.

## Wiring

After this note matures:

- `nextflow-summary-to-galaxy-interface` adds a `references[]` entry of `kind: research`, `mode: condense`, `load: on-demand`, with a trigger like "When the source pipeline declares iGenomes-derived params or per-asset reference path params."
- `nextflow-summary-to-galaxy-data-flow` does the same for the dbkey / index-handling parts.
- `summarize-nextflow`'s eval.md grows a case for "every iGenomes-derived param is captured in `params[]` with the dynamic-source description, even though it's absent from `nextflow_schema.json`."

## Out of scope for this issue

- Authoring the WES/WGS sibling-vs-flag rule. That's a separate gap from the same refinement journal — track in a sibling issue.
- Implementing Galaxy tool-wrapper rebuild behavior. The v1 posture *relies* on existing IWC wrappers already doing this; if they don't for some asset, that's a wrapper bug, not a translation Mold concern.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.