nf-core / nf-core/tools

Auto-updated markdown table of default resource requests in pipeline template

Open
#4,345 6 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
322
Forks
255
Avg merge
2d 3h
Merged PRs (30d)
5

Description

Description of feature

A common question a newcomer to a pipeline may ask is how much computing resources (memory/CPUs) a pipeline run will need (at least on default).

Currently this information is buried across multiple places: conf/base.config for values, modules/**/*.nf for labels etc, making it hard for a user to find this information.

I propose that we add a markdown table to pipeline docs that summarises this information:

  • Name of module
  • How much CPU/memory/time it is requested
  • Where is the source of this definition (withName? withLabel? defaults?)

This would make it easier for people to get an idea of what they would normally expect to need.

Technical considerations

This would need to be included in e.g. a linting action to be auto-updated, as I suspect it would be regularly forgotten to be updated when a user adds/tweaks a module.

The rules are quite straight forward to assess this based on Nextflow configuration order: withName > withLabel > default, so it is better to do this rather than have an AI LLM (which was slow when I did it once, and made a lot of mistakes - so wouldn't be worth the environmental impact).

The code that sweeps the code bases will need to be careful to also check for module aliases and report these in the table, not the 'installed' module itself.

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing conf/base.config and modules/**/*.nf to understand the resource values, labels, module names, and aliases that must be summarised. Then inspect the existing documentation and linting action entry points; done means an automatically maintained Markdown table reports module resources and their configuration source, including aliases and precedence.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown, python
Domain
ci-cd, documentation, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.