nf-core pipelines download does not work with SSLError
Open
Nobody has claimed this yet.
bug
download
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Description of the bug
I use nf-core/tools version 3.1.1 to download pipelines, but failed.
Command used and terminal output
nf-core pipelines download -r 3.18.0 -o ./pipelines -x tar.gz -f -c yes -s singularity rnaseq
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 3.1.1 - https://nf-co.re
WARNING Could not find GitHub authentication token. Some API requests may fail.
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests/adapters.py:667 in send │
│ │
│ 664 │ │ │ timeout = TimeoutSauce(connect=timeout, read=timeout) │
│ 665 │ │ │
│ 666 │ │ try: │
│ ❱ 667 │ │ │ resp = conn.urlopen( │
│ 668 │ │ │ │ method=request.method, │
│ 669 │ │ │ │ url=url, │
│ 670 │ │ │ │ body=request.body, │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/urllib3/connectionpool.py:841 in urlopen │
│ │
│ 838 │ │ │ elif isinstance(new_e, (OSError, HTTPException)): │
│ 839 │ │ │ │ new_e = ProtocolError("Connection aborted.", new_e) │
│ 840 │ │ │ │
│ ❱ 841 │ │ │ retries = retries.increment( │
│ 842 │ │ │ │ method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2] │
│ 843 │ │ │ ) │
│ 844 │ │ │ retries.sleep() │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/urllib3/util/retry.py:519 in increment │
│ │
│ 516 │ │ │
│ 517 │ │ if new_retry.is_exhausted(): │
│ 518 │ │ │ reason = error or ResponseError(cause) │
│ ❱ 519 │ │ │ raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type │
│ 520 │ │ │
│ 521 │ │ log.debug("Incremented Retry for (url='%s'): %r", url, new_retry) │
│ 522 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
MaxRetryError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/nf-core/rnaseq/branches?per_page=100 (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)')))
During handling of the above exception, another exception occurred:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/meng/miniconda3/bin/nf-core:8 in <module> │
│ │
│ 5 from nf_core.__main__ import run_nf_core │
│ 6 if __name__ == '__main__': │
│ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(run_nf_core()) │
│ 9 │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/__main__.py:182 in run_nf_core │
│ │
│ 179 │ │ │ log.debug(f"Could not check latest version: {e}") │
│ 180 │ │ stderr.print("\n") │
│ 181 │ # Launch the click cli │
│ ❱ 182 │ nf_core_cli(auto_envvar_prefix="NFCORE") │
│ 183 │
│ 184 │
│ 185 @tui( │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/rich_click/rich_command.py:367 in __call__ │
│ │
│ 364 │ │ # Include this here because I run into a false warning │
│ 365 │ │ # in the PyCharm IDE otherwise; for some reason PyCharm doesn't │
│ 366 │ │ # seem to think RichGroups are callable. (No issues with Mypy, though.) │
│ ❱ 367 │ │ return super().__call__(*args, **kwargs) │
│ 368 │
│ 369 │
│ 370 class RichCommandCollection(CommandCollection, RichGroup): │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/click/core.py:1161 in __call__ │
│ │
│ 1158 │ │
│ 1159 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │
│ 1160 │ │ """Alias for :meth:`main`.""" │
│ ❱ 1161 │ │ return self.main(*args, **kwargs) │
│ 1162 │
│ 1163 │
│ 1164 class Command(BaseCommand): │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/rich_click/rich_command.py:152 in main │
│ │
│ 149 │ │ try: │
│ 150 │ │ │ try: │
│ 151 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │
│ ❱ 152 │ │ │ │ │ rv = self.invoke(ctx) │
│ 153 │ │ │ │ │ if not standalone_mode: │
│ 154 │ │ │ │ │ │ return rv │
│ 155 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/click/core.py:1697 in invoke │
│ │
│ 1694 │ │ │ │ super().invoke(ctx) │
│ 1695 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │
│ 1696 │ │ │ │ with sub_ctx: │
│ ❱ 1697 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │
│ 1698 │ │ │
│ 1699 │ │ # In chain mode we create the contexts step by step, but after the │
│ 1700 │ │ # base command has been invoked. Because at that point we do not │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/click/core.py:1697 in invoke │
│ │
│ 1694 │ │ │ │ super().invoke(ctx) │
│ 1695 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │
│ 1696 │ │ │ │ with sub_ctx: │
│ ❱ 1697 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │
│ 1698 │ │ │
│ 1699 │ │ # In chain mode we create the contexts step by step, but after the │
│ 1700 │ │ # base command has been invoked. Because at that point we do not │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/click/core.py:1443 in invoke │
│ │
│ 1440 │ │ │ echo(style(message, fg="red"), err=True) │
│ 1441 │ │ │
│ 1442 │ │ if self.callback is not None: │
│ ❱ 1443 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ 1444 │ │
│ 1445 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]: │
│ 1446 │ │ """Return a list of completions for the incomplete value. Looks │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/click/core.py:788 in invoke │
│ │
│ 785 │ │ │
│ 786 │ │ with augment_usage_errors(__self): │
│ 787 │ │ │ with ctx: │
│ ❱ 788 │ │ │ │ return __callback(*args, **kwargs) │
│ 789 │ │
│ 790 │ def forward(__self, __cmd: "Command", *args: t.Any, **kwargs: t.Any) -> t.Any: │
│ 791 │ │ """Similar to :meth:`invoke` but fills in default keyword │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/click/decorators.py:33 in new_func │
│ │
│ 30 │ """ │
│ 31 │ │
│ 32 │ def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R": │
│ ❱ 33 │ │ return f(get_current_context(), *args, **kwargs) │
│ 34 │ │
│ 35 │ return update_wrapper(new_func, f) │
│ 36 │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/__main__.py:443 in │
│ command_pipelines_download │
│ │
│ 440 │ """ │
│ 441 │ Download a pipeline, nf-core/configs and pipeline singularity images. │
│ 442 │ """ │
│ ❱ 443 │ pipelines_download( │
│ 444 │ │ ctx, │
│ 445 │ │ pipeline, │
│ 446 │ │ revision, │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/commands_pipelines.py:203 in │
│ pipelines_download │
│ │
│ 200 │ │ container_cache_index, │
│ 201 │ │ parallel_downloads, │
│ 202 │ ) │
│ ❱ 203 │ dl.download_workflow() │
│ 204 │
│ 205 │
│ 206 # nf-core pipelines create-params-file │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/pipelines/download.py:182 in │
│ download_workflow │
│ │
│ 179 │ │ # Get workflow details │
│ 180 │ │ try: │
│ 181 │ │ │ self.prompt_pipeline_name() │
│ ❱ 182 │ │ │ self.pipeline, self.wf_revisions, self.wf_branches = nf_core.utils.get_repo_ │
│ 183 │ │ │ │ self.pipeline, self.wfs │
│ 184 │ │ │ ) │
│ 185 │ │ │ self.prompt_revision() │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/utils.py:1086 in │
│ get_repo_releases_branches │
│ │
│ 1083 │ │ │ raise AssertionError(f"Not able to find pipeline '{pipeline}'") │
│ 1084 │ │
│ 1085 │ # Get branch information from github api - should be no need to check if the repo ex │
│ ❱ 1086 │ branch_response = gh_api.safe_get(f"https://api.github.com/repos/{pipeline}/branches │
│ 1087 │ for branch in branch_response.json(): │
│ 1088 │ │ if ( │
│ 1089 │ │ │ branch["name"] != "TEMPLATE" │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/utils.py:599 in safe_get │
│ │
│ 596 │ │ """ │
│ 597 │ │ if not self.has_init: │
│ 598 │ │ │ self.lazy_init() │
│ ❱ 599 │ │ request = self.get(url) │
│ 600 │ │ if request.status_code in self.return_retry: │
│ 601 │ │ │ stderr = rich.console.Console(stderr=True, force_terminal=rich_force_colors( │
│ 602 │ │ │ try: │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/nf_core/utils.py:620 in get │
│ │
│ 617 │ │ """ │
│ 618 │ │ if not self.has_init: │
│ 619 │ │ │ self.lazy_init() │
│ ❱ 620 │ │ return super().get(url, **kwargs) │
│ 621 │ │
│ 622 │ def request_retry(self, url, post_data=None): │
│ 623 │ │ """ │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests_cache/session.py:127 in get │
│ │
│ 124 │ # Wrapper methods to add return type hints │
│ 125 │ def get(self, url: str, params=None, **kwargs) -> AnyResponse: # type: ignore │
│ 126 │ │ kwargs.setdefault('allow_redirects', True) │
│ ❱ 127 │ │ return self.request('GET', url, params=params, **kwargs) │
│ 128 │ │
│ 129 │ def options(self, url: str, **kwargs) -> AnyResponse: # type: ignore │
│ 130 │ │ kwargs.setdefault('allow_redirects', True) │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests_cache/session.py:183 in request │
│ │
│ 180 │ │ """ │
│ 181 │ │ headers = set_request_headers(headers, expire_after, only_if_cached, refresh, fo │
│ 182 │ │ with patch_form_boundary() if kwargs.get('files') else nullcontext(): │
│ ❱ 183 │ │ │ return super().request(method, url, *args, headers=headers, **kwargs) # typ │
│ 184 │ │
│ 185 │ def send( │
│ 186 │ │ self, │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests/sessions.py:589 in request │
│ │
│ 586 │ │ │ "allow_redirects": allow_redirects, │
│ 587 │ │ } │
│ 588 │ │ send_kwargs.update(settings) │
│ ❱ 589 │ │ resp = self.send(prep, **send_kwargs) │
│ 590 │ │ │
│ 591 │ │ return resp │
│ 592 │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests_cache/session.py:230 in send │
│ │
│ 227 │ │ elif actions.resend_request: │
│ 228 │ │ │ response = self._resend(request, actions, cached_response, **kwargs) # type │
│ 229 │ │ elif actions.send_request: │
│ ❱ 230 │ │ │ response = self._send_and_cache(request, actions, cached_response, **kwargs) │
│ 231 │ │ else: │
│ 232 │ │ │ response = cached_response # type: ignore # Guaranteed to be non-None by t │
│ 233 │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests_cache/session.py:254 in │
│ _send_and_cache │
│ │
│ 251 │ │ If applicable, also handle conditional requests. │
│ 252 │ │ """ │
│ 253 │ │ request = actions.update_request(request) │
│ ❱ 254 │ │ response = super().send(request, **kwargs) │
│ 255 │ │ actions.update_from_response(response) │
│ 256 │ │ │
│ 257 │ │ if not actions.skip_write: │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests/sessions.py:703 in send │
│ │
│ 700 │ │ start = preferred_clock() │
│ 701 │ │ │
│ 702 │ │ # Send the request │
│ ❱ 703 │ │ r = adapter.send(request, **kwargs) │
│ 704 │ │ │
│ 705 │ │ # Total elapsed time of the request (approximately) │
│ 706 │ │ elapsed = preferred_clock() - start │
│ │
│ /home/meng/miniconda3/lib/python3.12/site-packages/requests/adapters.py:698 in send │
│ │
│ 695 │ │ │ │
│ 696 │ │ │ if isinstance(e.reason, _SSLError): │
│ 697 │ │ │ │ # This branch is for urllib3 v1.22 and later. │
│ ❱ 698 │ │ │ │ raise SSLError(e, request=request) │
│ 699 │ │ │ │
│ 700 │ │ │ raise ConnectionError(e, request=request) │
│ 701 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
SSLError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/nf-core/rnaseq/branches?per_page=100 (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)')))
System information
Nextflow version 24.10.2 and 22.10.6
Hardware Desktop
Executor local,
OS Ubuntu 20.04.5 LTS
Version of nf-core/tools 3.1.1
Python version 3.12
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the documented nf-core pipelines download command and tracing the request to api.github.com/repos/nf-core/rnaseq/branches. Inspect how the CLI handles the reported requests/urllib3 certificate error; done means pipeline downloads no longer fail with this SSLError and the behavior is covered by the relevant tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, python
- Domain
- api, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100