MultiQC results not cacheable
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Description of the bug
Within the demux pipeline, we observe that MultiQC is never cached and thus always reexecuted, see also https://github.com/nf-core/demultiplex/issues/233 for a full discussion. Some pipelines configure in the conf/modules.config that MultiQC should not be cached at all (by setting caching = false), whereas this is not desired and thus also not configured in demultiplex as such.
The main issue boils down to some files being created using the official nf-core tools template tooling, e.g. workflow_summary_mqc.yaml and methods_description_mqc.yaml by changing whats ingested into these two YAML files as well as the sorting of these files / content that is in these files. The latter can be addressed by some channel magic, the former is more difficult to address and should thus be tackled probably here within the template and then auto-fixed via a template merge across multiple pipelines.
If people then actively turn off caching thats their choice - but at the moment its broken for those who would like not to re-run steps with MultiQC involved unfortunately :)
Command used and terminal output
nf-core run nf-core/demultiplexing -profile test,docker --outdir fancy_dir
System information
Doesnt really matter here
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the nf-core tools template code that generates workflow_summary_mqc.yaml and methods_description_mqc.yaml, then review the linked demultiplex discussion. Compare how the YAML contents and ordering affect MultiQC caching, and identify the template-level change needed so generated results remain cacheable without forcing caching=false across pipelines.
Written by the indexing model from the issue text.
Assessment
- Domain
- bioinformatics, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100