nextflow-io / nextflow-io/nextflow
Support passing multiple values to a single input in nextflow module run
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
New feature
When running nextflow module run, there is currently no ergonomic way to pass multiple values for a single input parameter (path, val). For path inputs the only working approach is a glob pattern, which is fragile and not always applicable. For val and other input types there is no workaround at all.
Use case
When using nextflow module run to test or run a module that accepts multiple files for a single input (e.g. path(csv, stageAs: 'inputs/csv*/*') in nf-core/csvtk/concat), users naturally reach for one of the following CLI forms:
Space-separated values:
nextflow module run nf-core/csvtk/concat \
--csv sample_a.csv sample_b.csv \
--in_format csv --out_format csv --outdir results
Repeated flag:
nextflow module run nf-core/csvtk/concat \
--csv sample_a.csv \
--csv sample_b.csv \
--in_format csv --out_format csv --outdir results
Both forms run without error but silently pass only one file to the process — the concatenation produces a copy of the first file rather than a combined result. There is no warning to the user.
The only currently working approach is a glob pattern:
nextflow module run nf-core/csvtk/concat \
--csv "/path/to/sample_*.csv" \
--in_format csv --out_format csv --outdir results
This works because Nextflow resolves the glob into a list via files(), but it triggers a WARN: The file() function was called with a glob pattern that matched a collection of files -- use files() instead. warning, requires files to share a naming pattern, and fails entirely when files live in different directories.
Suggested implementation
-
Support space-separated multi-value
pathparameters innextflow module run: when a parameter is declared aspathtype inmeta.ymland multiple space-separated values are provided, resolve each token as a file and pass them as a list. -
Support repeated flags for
pathparameters:--csv a.csv --csv b.csvshould accumulate into a list rather than overwriting. -
Emit a warning when a
pathparameter resolves to a single file but the process input declares it accepts multiple files (e.g. viastageAspatterns likeinputs/csv*/*), to surface the silent-truncation bug users currently experience.
Either (1) or (2) would resolve the main issue; (3) is a safety net for the current behaviour.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the nextflow module run CLI input handling and the meta.yml declarations for path parameters. Reproduce the space-separated and repeated-flag examples, then trace how values are resolved and passed to the process. Done means supported multi-value input forms accumulate all files, with the requested warning behavior covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100