Re-test and label/exclude modules with failing tests
@mashehu is already working on this.
Since Jan 30, 2026.
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Problem
When a widely-used module (e.g., untar, samtools/index) is modified, it triggers tests for all dependent modules and subworkflows. This frequently surfaces pre-existing test failures in modules that haven't been tested in a while, blocking PRs that are unrelated to those failures.
This creates a frustrating situation where contributors end up debugging and fixing other people's broken tests just to get their own changes merged, or requiring maintainers to override CI failures.
Recent example: https://github.com/nf-core/modules/pull/9796 - a topic channel migration for 5 modules triggered failures in galah, kraken subworkflows, and others that were already broken.
Proposed Solution
As agreed in the nf-core maintainers meeting, implement a periodic CI job (weekly/monthly?) that:
- Runs all module/subworkflow tests regardless of changes
- Labels modules with failing tests (e.g.,
tests-failing,needs-fix) - Optionally excludes known-failing modules from PR CI runs until fixed
This would:
- Surface broken tests proactively rather than when someone touches a dependency
- Make it clear which modules need attention
- Prevent unrelated failures from blocking PRs
- Provide a "health dashboard" for the modules repository
Additional Considerations
- Labels could trigger notifications to module maintainers
- A scheduled issue could be auto-created listing all failing modules
- Consider a "quarantine" mechanism where known-failing tests are skipped in PR CI but tracked separately
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.