nf-core / nf-core/modules

[FEATURE] Add a new label for internet-connected modules

Open
#1,783 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement wishlist
Dominant language
Nextflow
Stars
429
Forks
1.1k
Avg merge
1d 6h
Merged PRs (30d)
153

Description

Is your feature request related to a problem? Please describe

For HPC systems where compute nodes do not have internet access (examples: tars, UPPMAX, Atlas), modules that attempt to contact the internet fail when launched by Nextflow using the job scheduler. This includes modules such as

Describe the solution you'd like

I suggest a new label -- something like process-requires-internet -- that goes on these modules requiring internet connectivity. Config designers could then ensure all internet-connected processes are run on nodes with internet access, or fail a pipeline containing one of these modules early.

Describe alternatives you've considered

Alternative: remove all internet-connected processes from this repository and put a statement in the docs that all processes must be able to be run offline. This is probably best from a pipeline reproducibility standpoint, but less practical from my point of view.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked ncbigenomedownload module, the proposed edirect/esearch issue, and the referenced HPC configuration examples. Determine how a process-requires-internet label should be represented and where it must be applied; done means affected modules are identifiable so configuration designers can handle or reject them early.

Written by the indexing model from the issue text.

Assessment

Domain
hpc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.