nf-core / nf-core/modules

The `fastp` module will zero source FASTQ files if they end in `.fastp.fastq.gz` (data loss scenario)

Open
#7,961 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Nextflow
Stars
429
Forks
1.1k
Avg merge
1d 6h
Merged PRs (30d)
153

Description

Have you checked the docs?
Description of the bug

This is due to the logic here:

https://github.com/nf-core/modules/blob/ee5b3472df499c78a4cc3f6b57c4da7cb4d75d25/modules/nf-core/fastp/main.nf#L34

And due to the symlink done here:

https://github.com/nf-core/modules/blob/ee5b3472df499c78a4cc3f6b57c4da7cb4d75d25/modules/nf-core/fastp/main.nf#L82

fastp will get differently-named inputs and outputs, and will end up overwriting (zeroing) the source file. This is a scenario that leads to data loss.

Command used and terminal output
1. Create FASTQ files like `test1_R1.fastp.fastq.gz` and `test_R2.fastp.fastq.gz`. 
2. Run the `fastp` module using these files as input
3. Check the original files
4. The original files  have now 0 bytes in size.
Relevant files

No response

System information

Nextflow 24.10.4
Hardware: HPC
Executor: slurm
Container engine: apptainer
OS: Debian 10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect modules/nf-core/fastp/main.nf at the linked lines 34 and 82, then reproduce the issue with input files ending in .fastp.fastq.gz using the stated Nextflow setup. Done means the module produces its outputs without changing the original FASTQ files, which should retain their contents and nonzero size.

Written by the indexing model from the issue text.

Assessment

Domain
bioinformatics
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.