nf-core / nf-core/tools

Unhelpful error message in nf-core download when container registry is not defined or is wrong

Open
#2,892 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug download
Dominant language
Python
Stars
322
Forks
255
Avg merge
2d 3h
Merged PRs (30d)
5

Description

Description of the bug

Hi! I was trying to download a non nf-core nextflow pipeline using nf-core download, which I understand could (or should?) work. However when I tried this for epi2me-labs/wf-single-cell version 1.1.0 it would fail in downloading the singularity images. I could gather after carefully reading the commandline output that it tried to download the images from quay.io when in reality the images are hosted on docker.io. The error message was not helpful in immediately identifying this as it only stated that it "failed for unclear reasons.".

This might be caused by the authors not specifying docker.io anywhere e.g. in docker.registry. I was able to get around this problem by using the option -l docker.io (thanks @ewels). This could be less troublesome by a more informative error message and/or not defaulting to quay.io if no registry is given - I'll leave that discussion to others.

Thanks.

Command used and terminal output
$ nf-core download epi2me-labs/wf-single-cell
[...]
INFO     Saving 'epi2me-labs/wf-single-cell'                                                                                                                                                                                                  
          Pipeline revision: 'v1.1.0'                                                                                                                                                                                                         
          Use containers: 'singularity'                                                                                                                                                                                                       
          Container library: 'quay.io'                                                                                                                                                                                                        
          Output directory: 'epi2me-labs-wf-single-cell_v1.1.0'                                                                                                                                                                               
          Include default institutional configuration: 'True'                                                                                                                                                                                 
INFO     Downloading centralised configs from GitHub                                                                                                                                                                                          
INFO     Downloading workflow files from GitHub                                                                                                                                                                                               
INFO     Processing workflow revision v1.1.0, found 2 container images in total.                                                                                                                                                              
Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% • 0/2 completed
ontresearch/wf-common:sha2816439fd5aa81902836ea5794447b54947fa056  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR    "ontresearch/wf-common:sha2816439fd5aa81902836ea5794447b54947fa056" failed for unclear reasons.                                                                                                                                      
                                                                                                                                                                                                                                              
INFO     Pulling of "ontresearch/wf-common:sha2816439fd5aa81902836ea5794447b54947fa056" failed.                                                                                                                                               
         Please troubleshoot the command                                                                                                                                                                                                      
         "singularity pull --name /crex/proj/sllstore2017079/private/remi/tmp/epi2me-labs-wf-single-cell_v1.1.0/singularity-images/ontresearch-wf-common-sha2816439fd5aa81902836ea5794447b54947fa056.img                                      
         docker://quay.io/ontresearch/wf-common:sha2816439fd5aa81902836ea5794447b54947fa056" manually.                                                                                                                                        
                                                                                                                                                                                                                                              
ERROR    "ontresearch/wf-common:sha2816439fd5aa81902836ea5794447b54947fa056" failed for unclear reasons.  
[...]
System information
  • nextflow version 23.04.3.5875
  • CentOS Linux 7
  • nf-core/tools 2.11.1
  • Python 3.10.6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the nf-core download command path that selects the container registry and reports failed Singularity pulls. Reproduce the example using epi2me-labs/wf-single-cell v1.1.0 and compare the behavior when the registry is absent or wrong. Done should provide an informative error identifying the registry problem, or clearly define and implement the intended default behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.