Picard "--TMP_DIR" arguments are hardcoded
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 230
- Forks
- 142
- Avg merge
- 13h 41m
- Merged PRs (30d)
- 1
Description
Description of the bug
An error "No space left on device" occurred by process NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES (E145I_REP1). Possible reason "--TMP_DIR" arguments are hardcoded in modules.config (example https://github.com/nf-core/atacseq/blob/1a1dbe52ffbd82256c941a032b0e22abbd925b8a/conf/modules.config#L358).
Caused by:
Process NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES (E145I_REP1) terminated with an error exit status (1)
Command executed:
picard \
-Xmx29491M \
MarkDuplicates \
--ASSUME_SORTED true --REMOVE_DUPLICATES false --VALIDATION_STRINGENCY LENIENT --TMP_DIR tmp \
--INPUT E145I_REP1.mLb.sorted.bam \
--OUTPUT E145I_REP1.mLb.mkD.sorted.bam \
--REFERENCE_SEQUENCE Mus_musculus.GRCm38.dna.primary_assembly.fa \
--METRICS_FILE E145I_REP1.mLb.mkD.sorted.MarkDuplicates.metrics.txt
cat <<-END_VERSIONS > versions.yml
"NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES":
picard: $(echo $(picard MarkDuplicates --version 2>&1) | grep -o 'Version:.*' | cut -f2- -d:)
END_VERSIONS
Command error:
Runtime.totalMemory()=4311744512
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" htsjdk.samtools.SAMException: Error loading new map from disk.
at htsjdk.samtools.CoordinateSortedPairInfoMap.ensureSequenceLoaded(CoordinateSortedPairInfoMap.java:143)
at htsjdk.samtools.CoordinateSortedPairInfoMap.remove(CoordinateSortedPairInfoMap.java:86)
at picard.sam.markduplicates.util.DiskBasedReadEndsForMarkDuplicatesMap.remove(DiskBasedReadEndsForMarkDuplicatesMap.java:61)
at picard.sam.markduplicates.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:560)
at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:270)
at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:289)
at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:104)
at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:114)
Caused by: java.io.IOException: No space left on device
at java.base/java.io.FileOutputStream.writeBytes(Native Method)
at java.base/java.io.FileOutputStream.write(FileOutputStream.java:349)
at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:142)
at java.base/java.io.FilterOutputStream.close(FilterOutputStream.java:182)
at htsjdk.samtools.CoordinateSortedPairInfoMap.ensureSequenceLoaded(CoordinateSortedPairInfoMap.java:119)
Command used and terminal output
Slurm declaration
export TMPDIR=$PWD
export SINGULARITY_TMPDIR=$PWD
export SINGULARITY_CACHEDIR=$PWD
unset XDG_RUNTIME_DIR
Relevant files
No response
System information
- Nextflow version - 24.01.0-edge
- Hardware - HPC
- Executor - slurm
- Container engine: - Singularity
- OS - Linux
- Version of nf-core/atacseq 2.1.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the referenced conf/modules.config entry around line 358 and the Slurm environment variables shown in the report. Check how the Picard MarkDuplicates process selects its temporary directory and reproduce the failure on an HPC job if possible. Done means the hardcoded temporary location no longer causes the reported device-space failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- bioinformatics, hpc
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100