1.13.0: `--nf_core_pipeline mag` emits a samplesheet mag rejects (empty `group`, `fastq_1/2` instead of `short_reads_1/2`)
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 202
- Forks
- 93
- Avg merge
- 1d 18m
- Merged PRs (30d)
- 15
Description
Description of the bug
--nf_core_pipeline mag, added in 1.13.0, emits a samplesheet that nf-core/mag rejects.
In subworkflows/local/channel_sra_create_csv/main.nf, buildPipelineMap adds these extras to the base row of sample, fastq_1, fastq_2:
mag: [group: '', short_reads_platform: 'ILLUMINA', long_reads_platform: ''],
Checked against mag 5.5.0's assets/schema_input.json, there are three problems:
1. group is required and is written empty. mag's schema has "required": ["sample", "group"] with group matching ^\S+$. An empty string fails that pattern, so validation fails on every row.
2. The read columns have the wrong names. mag reads short_reads_1 / short_reads_2 (and short_reads_1 has short_reads_platform as dependentRequired). fetchngs emits fastq_1 / fastq_2. Because mag only requires sample and group, this does not fail loudly on its own — rows would validate as samples with no reads attached.
3. short_reads_platform is hardcoded ILLUMINA rather than taken from the instrument_platform metadata column fetchngs already has. ENA uses mag's own spellings, so copying it across works directly, and a non-Illumina accession is currently mislabelled rather than rejected.
--nf_core_pipeline ampliseq and --nf_core_pipeline metatdenovo both look correct against their targets' schemas — this is only the mag target.
Suggested fix
mag: [group: '', short_reads_1: meta.fastq_1, short_reads_2: meta.fastq_2, short_reads_platform: meta.instrument_platform],
…with the base fastq_1/fastq_2 dropped for this target, and group either exposed as a parameter or documented as a column the user must fill in before running mag. group is a study-design decision (it drives co-abundance binning), so no upstream pipeline can infer it — which is exactly why defaulting it to '' produces an invalid sheet rather than a merely incomplete one.
Related
nf-core/detaxizer has the same empty-group defect in its own generated mag samplesheet, filed as nf-core/detaxizer#100. Two pipelines independently emitting an invalid sheet for the same target suggests the samplesheet contract between pipelines is worth standardising rather than reimplementing per pipeline. Raising that separately with the nf-core meta-omics SIG.
Command used and terminal output
Found by reading 1.13.0's source and mag 5.5.0's schema, not from a failed run — the pilot that prompted it ran on 1.12.0, before the mag target existed. To reproduce, run fetchngs with --nf_core_pipeline mag and pass the emitted samplesheet.csv to mag 5.5.0; nf-schema fails on group.
System information
- fetchngs 1.13.0
- nf-core/mag 5.5.0 (
assets/schema_input.json)
Found while validating inter-pipeline samplesheet handoffs for the nf-core meta-omics SIG end-to-end use case.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in subworkflows/local/channel_sra_create_csv/main.nf at buildPipelineMap, then compare the generated row with nf-core/mag 5.5.0's assets/schema_input.json. Run fetchngs with --nf_core_pipeline mag and pass the emitted samplesheet.csv to mag to verify schema validation and read-column handling. Done means the generated sheet has valid group handling, mag-compatible read columns, and the correct instrument platform.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100