Adopt combined-index Bismark alignment modes / rethink the --multicore default (Bismark 3.1.0)
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 198
- Forks
- 181
- Avg merge
- 6d 17h
- Merged PRs (30d)
- 1
Description
Context
Since the Rust rewrite, the Bismark aligner offers combined-index alignment modes that are generally faster and lighter than the classic per-strand + --multicore model. The Bismark docs now give a per-library-type decision guide: Choosing an alignment mode.
methylseq currently drives alignment with the classic model (the bismark/align module auto-computes --multicore from task.cpus). With Bismark 3.1.0 now in the pipeline (#614), it's worth deciding whether/how methylseq should adopt the newer modes.
The recommendations (summary)
Combined modes are driven by Bowtie 2 threads (-p), and --multicore is rejected in combined mode.
| Library | ⚡ Fastest / least CPU | 🔒 Byte-identical to Perl |
|---|---|---|
| Directional | --combined_index (one pass, tune -p) — ~22–28% less CPU |
standard, 2 instances |
| Non-directional | --combined_index_sequential — fastest, leanest RAM (~11 GB), byte-identical to the parallel combined run |
standard, 4 instances |
| PBAT | --combined_index (one pass, tune -p) |
standard, 2 instances |
What adopting this would involve in methylseq
- A one-time combined-index build in genome prep (
bismark prepare --combined_genome; ~+1.3 GB, extra prep time). - Selecting the alignment flag per library type (directional/PBAT vs non-directional) in the alignment subworkflow.
- Replacing the
--multicoreresource model with a-p-based one (threads throughtask.cpus/ process labels).
Key decision — and why I'm opening this rather than just PR'ing it
The combined index for directional/PBAT is concordance-gated, not byte-identical to Perl v0.25.1 (≈1 read in 10⁴ placed differently but equally validly); only non-directional-sequential is byte-identical. So this changes default alignment output for the common case. That's a deliberate defaults change for a flagship pipeline, so I'd like to agree the shape first:
- Default flip vs opt-in? Make combined-index the default, or expose it as a parameter and keep the faithful (byte-identical) path as default?
- Per-library-type auto-selection in the subworkflow — reasonable, or too magic?
- Any concern about the extra combined-index build step / disk (sequential's small BGZF spill)?
Happy to implement whichever shape we land on. Follow-up to #614 (the straight 3.0.0→3.1.0 bump, which is intentionally byte-identical and separate from this).
cc @pinin4fjords
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Bismark Choosing an alignment mode guide, then inspect methylseq’s bismark/align module, genome-prep entry point, and alignment subworkflow. Resolve the default-versus-opt-in and per-library-type decisions before changing the combined-index preparation and replacing task.cpus-driven --multicore resources with -p. Done means the agreed mode, resource model, and extra index-build requirements are implemented consistently.
Written by the indexing model from the issue text.
Assessment
- Domain
- bioinformatics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100