AMPCOMBI2_PARSETABLES error during stage-out with process.stageOutMode = 'move' -- "mv: cannot stat 'XYZ': No such file or directory"
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 119
- Forks
- 40
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 5
Description
Description of the bug
I am consistently getting errors from AMPCOMBI2_PARSETABLES when running on a cluster with process.stageOutMode = 'move'. The issue seems to occur during stage-out and results in error messages like the following:
ERROR ~ Error executing process > 'NFCORE_FUNCSCAN:FUNCSCAN:AMP:AMPCOMBI2_PARSETABLES (sample_2)'
Caused by:
Process `NFCORE_FUNCSCAN:FUNCSCAN:AMP:AMPCOMBI2_PARSETABLES (sample_2)` terminated with an error exit status (1)
Command executed:
ampcombi parse_tables \
--path_list 'sample_2.ampir.tsv' 'sample_2.macrel.prediction' \
--faa sample_2_pyrodigal.faa \
--gbk sample_2_pyrodigal.gbk \
--sample_list sample_2 \
--amp_database DRAMP \
--amp_database_dir amp_DRAMP_database \
\
--aminoacid_length 120 --db_evalue 5 --amp_cutoff 0.6 --ampir_file .ampir.tsv --amplify_file .amplify.tsv --macrel_file .macrel.prediction --hmmsearch_file .hmmer_hmmsearch.txt --ampgram_file '.tsv' --amptransformer_file '.txt' --neubi_file '.fasta' --log TRUE --hmm_evalue 0.06 --window_size_stop_codon 60 --window_size_transporter 11 \
--threads 4
cat <<-END_VERSIONS > versions.yml
"NFCORE_FUNCSCAN:FUNCSCAN:AMP:AMPCOMBI2_PARSETABLES":
ampcombi: $(ampcombi --version | sed 's/ampcombi //')
END_VERSIONS
Command exit status:
1
Command output:
Success: 'ampcombi parse_tables' completed successfully ! Now run ampcombi complete !
Command error:
mv: cannot stat 'amp_DRAMP_database/general_amps_2026_04_05.fasta': No such file or directory
Work dir:
/data/share/users/amizeranschi/test-funcscan-error/work/66/f07f456eb57a447b5d07b451bb034e
Container:
quay.io/biocontainers/ampcombi:2.0.1--pyhdfd78af_0
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
-- Check '.nextflow.log' file for details
Below is a minimal set of commands that should reproduce this error on a Slurm cluster, using the test profile:
cat << \EOF > custom.config
process.executor = 'slurm'
process.scratch = 'ram-disk'
process.stageInMode = 'copy'
process.stageOutMode = 'move'
process.errorStrategy = 'retry'
process.maxRetries = 2
EOF
rm -rf .nextflow* work test-ampcombi-error
clear
date
nextflow run nf-core/funcscan -r 60385fe \
-profile docker,test \
-c custom.config \
--run_amp_screening \
--amp_skip_amplify \
--run_arg_screening false \
--run_bgc_screening false \
--run_cazyme_screening false \
--run_taxa_classification false \
--run_protein_annotation false \
--outdir test-ampcombi-error
A local fix (which only deals with the symptom) is to set process.stageOutMode = 'copy' in the config. I have tested this and confirm that it "works". However, it would be more useful to fix this issue upstream in the module.
The error seems to be caused by overlapping outputs in the module, specifically here:
https://github.com/nf-core/funcscan/blob/dev/modules/nf-core/ampcombi2/parsetables/main.nf#L19
Both the main directory "${meta.id}/" and some of its content are declared as outputs. If process.stageOutMode = 'move' and the main directory is moved first, then Nextflow won't be able to subsequently stage-out the individual files as well, resulting in errors like the one above: mv: cannot stat 'XYZ': No such file or directory.
A similar issue is likely to occur with the "amp_${opt_amp_db}_database/" directory and its content:
https://github.com/nf-core/funcscan/blob/dev/modules/nf-core/ampcombi2/parsetables/main.nf#L26
Command used and terminal output
Relevant files
No response
System information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Inspect modules/nf-core/ampcombi2/parsetables/main.nf at the output declarations around lines 19 and 26, then reproduce with process.stageOutMode = 'move' using the provided test command. Check how the directory and its contents are staged out; done means the module no longer attempts to move overlapping outputs and the reproduction completes successfully.
Written by the indexing model from the issue text.
Assessment
- Domain
- bioinformatics, tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100