Linting a subworkflow meta YML with components not present in the workflow throws a linting error
Open
Nobody has claimed this yet.
bug
linting
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Description of the bug
As in title: if you include components in the meta.yml that are not present in the workflow (possible only when git remote is specified?), linting fails with a critical error:
DEBUG sys.platform='darwin', git_executable='git' util.py:494
DEBUG Popen(['git', 'version'], cwd=/Users/jd42/.config/nfcore/sanger-tol/nf-core-modules, stdin=None, shell=False, universal_newlines=False) cmd.py:1270
DEBUG Popen(['git', 'fetch', '-v', '--progress', '--', 'origin'], cwd=/Users/jd42/.config/nfcore/sanger-tol/nf-core-modules, stdin=None, shell=False, universal_newlines=True) cmd.py:1270
DEBUG Popen(['git', 'checkout', 'main'], cwd=/Users/jd42/.config/nfcore/sanger-tol/nf-core-modules, stdin=None, shell=False, universal_newlines=False) cmd.py:1270
DEBUG Popen(['git', 'merge', 'origin/main'], cwd=/Users/jd42/.config/nfcore/sanger-tol/nf-core-modules, stdin=None, shell=False, universal_newlines=False) cmd.py:1270
DEBUG Got '/Users/jd42/.config/nfcore/sanger-tol/nf-core-modules' as path utils.py:349
DEBUG No config cache found utils.py:396
DEBUG Running command: nextflow config -flat /Users/jd42/.config/nfcore/sanger-tol/nf-core-modules utils.py:448
DEBUG Could not open /Users/jd42/.config/nfcore/sanger-tol/nf-core-modules/main.nf to look for parameter declarations - [Errno 2] No such file or directory: utils.py:430
'/Users/jd42/.config/nfcore/sanger-tol/nf-core-modules/main.nf'
DEBUG Using config file: /Users/jd42/.config/nfcore/sanger-tol/nf-core-modules/.nf-core.yml utils.py:1481
DEBUG Got '.' as path utils.py:349
DEBUG No config cache found utils.py:396
DEBUG Running command: nextflow config -flat . utils.py:448
DEBUG Could not open main.nf to look for parameter declarations - [Errno 2] No such file or directory: 'main.nf' utils.py:430
DEBUG Using config file: .nf-core.yml utils.py:1481
DEBUG Got 'tests/config' as path utils.py:349
DEBUG Found a config cache, loading: /Users/jd42/.nextflow/nf-core/wf-config-cache-898cf05086b57f6f21815c39d.json utils.py:383
DEBUG Registry set to quay.io __init__.py:164
INFO Linting modules repo: '.' __init__.py:136
INFO Linting subworkflow: 'fasta_map_long_reads' __init__.py:140
DEBUG sys.platform='darwin', git_executable='git' util.py:494
Pulling from 'nf-core/modules' (https://github.com/nf-core/modules.git) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Waiting for responseDEBUG Popen(['git', 'version'], cwd=/Users/jd42/.config/nfcore/nf-core/modules, stdin=None, shell=False, universal_newlines=False) cmd.py:1270
DEBUG Popen(['git', 'fetch', '-v', '--progress', '--', 'origin'], cwd=/Users/jd42/.config/nfcore/nf-core/modules, stdin=None, shell=False, universal_newlines=True) cmd.py:1270
DEBUG Popen(['git', 'checkout', 'master'], cwd=/Users/jd42/.config/nfcore/nf-core/modules, stdin=None, shell=False, universal_newlines=False) cmd.py:1270
DEBUG Popen(['git', 'merge', 'origin/master'], cwd=/Users/jd42/.config/nfcore/nf-core/modules, stdin=None, shell=False, universal_newlines=False) cmd.py:1270
DEBUG Got '/Users/jd42/.config/nfcore/nf-core/modules' as path utils.py:349
DEBUG Found a config cache, loading: /Users/jd42/.nextflow/nf-core/wf-config-cache-9f4bce35ba6f9f759cbf54fa7.json utils.py:383
DEBUG Using config file: /Users/jd42/.config/nfcore/nf-core/modules/.nf-core.yml utils.py:1481
CRITICAL 'samtools/faidx' commands_subworkflows.py:147
Path to YAML: https://github.com/sanger-tol/nf-core-modules/blob/6b138b09f731e56f611a4cb1edd1ef57ce7b22e7/subworkflows/sanger-tol/fastx_map_long_reads/meta.yml
Path to main.nf: https://github.com/sanger-tol/nf-core-modules/blob/6b138b09f731e56f611a4cb1edd1ef57ce7b22e7/subworkflows/sanger-tol/fastx_map_long_reads/main.nf
Components section:
components:
- fastxalign/index
- fastxalign/minimap2align
- samtools/faidx:
git_remote: https://github.com/nf-core/modules.git
- samtools/merge:
git_remote: https://github.com/nf-core/modules.git
- samtools/markdup:
git_remote: https://github.com/nf-core/modules.git
- minimap2/index:
git_remote: https://github.com/nf-core/modules.git
Removing the components not included in the workflow fixes the critical error.
Command used and terminal output
nf-core --verbose subworkflows --git-remote https://github.com/sanger-tol/nf-core-modules.git lint fastx_map_long_reads
System information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with commands_subworkflows.py around line 147, then compare the referenced meta.yml and main.nf files to see how components are collected during subworkflow linting. Reproduce the issue with the provided nf-core command and verify that components absent from the workflow no longer produce a critical error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100