nf-core / nf-core/fetchngs

fetchngs test fails

Open
#336 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Nextflow
Stars
202
Forks
93
Avg merge
1d 18m
Merged PRs (30d)
15

Description

I trying the fetchngs pipeline but I can't even get the test to run. Any advice? see log below.

best,
sb

> nextflow run  nf-core/fetchngs -profile test,singularity --outdir test -resume 
Nextflow 24.10.4 is available - Please consider updating your version to it

 N E X T F L O W   ~  version 24.10.2

Launching `https://github.com/nf-core/fetchngs` [nice_jepsen] DSL2 - revision: 8ec2d934f9 [master]

WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/fetchngs v1.12.0-g8ec2d93
------------------------------------------------------
Core Nextflow options
  revision                  : master
  runName                   : nice_jepsen
  containerEngine           : singularity
  launchDir                 : /cluster/tmp/userx/data
  workDir                   : /cluster/tmp/userx/data/work
  projectDir                : /home/userx/.nextflow/assets/nf-core/fetchngs
  userName                  : userx
  profile                   : test,singularity
  configFiles               : 

Input/output options
  input                     : https://raw.githubusercontent.com/nf-core/test-datasets/2732b911c57e607fa7aea5ba0c3d91b25bafb662/testdata/v1.12.0/sra_ids_test.csv
  outdir                    : test

Institutional config options
  config_profile_name       : Test profile
  config_profile_description: Minimal test dataset to check pipeline function

Max job request options
  max_cpus                  : 2
  max_memory                : 6.GB
  max_time                  : 6.h

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/fetchngs for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.5070524

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/fetchngs/blob/master/CITATIONS.md
------------------------------------------------------
executor >  local (33)
[7d/b55df7] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (GSM4907283)                                                            [100%] 10 of 10, cached: 10 ✔
[88/6b13c0] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP (6)                                                                     [100%] 10 of 10, cached: 10 ✔
[aa/396e50] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)                                                     [ 66%] 22 of 33, failed: 22, retries: 22
[7e/0b4955] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_SRATOOLSNCBISETTINGS (ncbi-settings)   [100%] 1 of 1, cached: 1 ✔
[13/de5c48] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_PREFETCH (SRR14593545)               [100%] 3 of 3, cached: 3 ✔
[c0/7a1127] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_FASTERQDUMP (SRX9315476_SRR12848126) [100%] 3 of 3, cached: 3 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:ASPERA_CLI                                                                                 -
[ff/a1cfda] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET (SRX10940790_SRR14593545)                                               [100%] 3 of 3, cached: 3
[-        ] process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG                                                                    -
[7c/523396] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX1234253_ERR1160846)` terminated with an error exit status (4) -- Execution is retried (1)
[fd/42431b] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX026011_DRR028935)` terminated with an error exit status (4) -- Execution is retried (1)
[5b/260cf9] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055518)` terminated with an error exit status (4) -- Execution is retried (1)
[79/1e6937] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)` terminated with an error exit status (4) -- Execution is retried (1)
[ba/64d55e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)` terminated with an error exit status (4) -- Execution is retried (1)
[af/c24d1f] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055519)` terminated with an error exit status (4) -- Execution is retried (1)
[ce/0091ea] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX6725035_SRR9984183)` terminated with an error exit status (4) -- Execution is retried (1)
[d5/aa8d48] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX024467_DRR026872)` terminated with an error exit status (4) -- Execution is retried (1)
[37/3c55f5] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055517)` terminated with an error exit status (4) -- Execution is retried (1)
[be/3eff92] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709227_SRR21711856)` terminated with an error exit status (4) -- Execution is retried (1)
[8f/9abf84] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709228_SRR21711855)` terminated with an error exit status (4) -- Execution is retried (1)
[78/a0ac7c] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX1234253_ERR1160846)` terminated with an error exit status (4) -- Execution is retried (2)
[df/f1c395] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)` terminated with an error exit status (4) -- Execution is retried (2)
[54/fb07ac] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX026011_DRR028935)` terminated with an error exit status (4) -- Execution is retried (2)
[6e/9d170f] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055518)` terminated with an error exit status (4) -- Execution is retried (2)
[cf/8aa6ef] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055519)` terminated with an error exit status (4) -- Execution is retried (2)
[40/83ea1d] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055517)` terminated with an error exit status (4) -- Execution is retried (2)
[0f/963215] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709227_SRR21711856)` terminated with an error exit status (4) -- Execution is retried (2)
[ea/77963a] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX6725035_SRR9984183)` terminated with an error exit status (4) -- Execution is retried (2)
[a1/405f94] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709228_SRR21711855)` terminated with an error exit status (4) -- Execution is retried (2)
[1a/154c02] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX024467_DRR026872)` terminated with an error exit status (4) -- Execution is retried (2)
[d4/58f935] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)` terminated with an error exit status (4) -- Execution is retried (2)
ERROR ~ Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)'

Caused by:
  Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)` terminated with an error exit status (4)


Command executed:
executor >  local (33)
[7d/b55df7] process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (GSM4907283)                                                            [100%] 10 of 10, cached: 10 ✔
[88/6b13c0] process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP (6)                                                                     [100%] 10 of 10, cached: 10 ✔
[aa/396e50] process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)                                                     [ 75%] 25 of 33, failed: 25, retries: 22
[7e/0b4955] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_SRATOOLSNCBISETTINGS (ncbi-settings)   [100%] 1 of 1, cached: 1 ✔
[13/de5c48] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_PREFETCH (SRR14593545)               [100%] 3 of 3, cached: 3 ✔
[c0/7a1127] process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_FASTERQDUMP (SRX9315476_SRR12848126) [100%] 3 of 3, cached: 3 ✔
[-        ] process > NFCORE_FETCHNGS:SRA:ASPERA_CLI                                                                                 -
[ff/a1cfda] process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET (SRX10940790_SRR14593545)                                               [100%] 3 of 3, cached: 3
[-        ] process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG                                                                    -
[7c/523396] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX1234253_ERR1160846)` terminated with an error exit status (4) -- Execution is retried (1)
[fd/42431b] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX026011_DRR028935)` terminated with an error exit status (4) -- Execution is retried (1)
[5b/260cf9] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055518)` terminated with an error exit status (4) -- Execution is retried (1)
[79/1e6937] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)` terminated with an error exit status (4) -- Execution is retried (1)
[ba/64d55e] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)` terminated with an error exit status (4) -- Execution is retried (1)
[af/c24d1f] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055519)` terminated with an error exit status (4) -- Execution is retried (1)
[ce/0091ea] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX6725035_SRR9984183)` terminated with an error exit status (4) -- Execution is retried (1)
[d5/aa8d48] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX024467_DRR026872)` terminated with an error exit status (4) -- Execution is retried (1)
[37/3c55f5] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055517)` terminated with an error exit status (4) -- Execution is retried (1)
[be/3eff92] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709227_SRR21711856)` terminated with an error exit status (4) -- Execution is retried (1)
[8f/9abf84] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709228_SRR21711855)` terminated with an error exit status (4) -- Execution is retried (1)
[78/a0ac7c] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX1234253_ERR1160846)` terminated with an error exit status (4) -- Execution is retried (2)
[df/f1c395] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)` terminated with an error exit status (4) -- Execution is retried (2)
[54/fb07ac] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX026011_DRR028935)` terminated with an error exit status (4) -- Execution is retried (2)
[6e/9d170f] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055518)` terminated with an error exit status (4) -- Execution is retried (2)
[cf/8aa6ef] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055519)` terminated with an error exit status (4) -- Execution is retried (2)
[40/83ea1d] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055517)` terminated with an error exit status (4) -- Execution is retried (2)
[0f/963215] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709227_SRR21711856)` terminated with an error exit status (4) -- Execution is retried (2)
[ea/77963a] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX6725035_SRR9984183)` terminated with an error exit status (4) -- Execution is retried (2)
[a1/405f94] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX17709228_SRR21711855)` terminated with an error exit status (4) -- Execution is retried (2)
[1a/154c02] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (DRX024467_DRR026872)` terminated with an error exit status (4) -- Execution is retried (2)
[d4/58f935] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)` terminated with an error exit status (4) -- Execution is retried (2)
ERROR ~ Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)'

Caused by:
  Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9504942_SRR13055520)` terminated with an error exit status (4)


Command executed:

  wget \
      -t 5 -nv -c -T 60 \
      -O SRX9504942_SRR13055520_1.fastq.gz \
      ftp.sra.ebi.ac.uk/vol1/fastq/SRR130/020/SRR13055520/SRR13055520_1.fastq.gz
  
  echo "795d460abff6009c7f5ff7c14b773fbe  SRX9504942_SRR13055520_1.fastq.gz" > SRX9504942_SRR13055520_1.fastq.gz.md5
  md5sum -c SRX9504942_SRR13055520_1.fastq.gz.md5
  
  wget \
      -t 5 -nv -c -T 60 \
      -O SRX9504942_SRR13055520_2.fastq.gz \
      ftp.sra.ebi.ac.uk/vol1/fastq/SRR130/020/SRR13055520/SRR13055520_2.fastq.gz
  
  echo "c3f34379356b28d6dac7212abc1a0a04  SRX9504942_SRR13055520_2.fastq.gz" > SRX9504942_SRR13055520_2.fastq.gz.md5
  md5sum -c SRX9504942_SRR13055520_2.fastq.gz.md5
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP":
      wget: $(echo $(wget --version | head -n 1 | sed 's/^GNU Wget //; s/ .*$//'))
  END_VERSIONS

Command exit status:
  4

Command output:
  (empty)

Command error:
  WARNING: Skipping mount /sw/rl9c/singularity/3.9.7/rl9_go1.19.4/install/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  wget: unable to resolve host address 'ftp.sra.ebi.ac.uk'

Work dir:
  /cluster/tmp/userx/data/work/d5/9a9851d83763d2de95a9e7d6a6d42c

Container:
  /cluster/tmp/userx/data/work/singularity/depot.galaxyproject.org-singularity-wget-1.20.1.img

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the nf-core/fetchngs test profile and the failed SRA_FASTQ_FTP processes in the provided run, focusing on the repeated exit status 4 failures. Re-run the shown test command with the singularity profile and determine what change or environment condition is needed for the test workflow to complete.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.