nf-core / nf-core/fetchngs

Check if running offline

Open
#18 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Nextflow
Stars
202
Forks
93
Avg merge
1d 18m
Merged PRs (30d)
15

Description

This pipeline won't work in offline situations because it needs to download metadata and fastq files from the web. We should probably add a check/warning/fail to make sure that there is an internet connection when running. This would have to factor in situations where login nodes could have an internet connection but the nodes on the cluster may be offline. We should be able to add try/catch logic to the bin/sra_ids_to_runinfo.py script to detect this or maybe we just write a separate module for this that can be re-used if required as this will only ever by run once at the beginning of the workflow with almost zero resources.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with bin/sra_ids_to_runinfo.py and trace where metadata and FASTQ downloads begin. Run the workflow in an environment where login and cluster nodes differ in internet access; done means the pipeline detects unavailable connectivity on the execution nodes and gives a clear warning or failure before downloads begin.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.