Disable k-mer heuristic in some cases

Open
#685 34 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python

Research direction

Start with the discussion carried over from issue #663 and review how the k-mer heuristic affects anchored, file-based adapters such as -g ^file:adapters1.fa -G ^file:adapters2.fa. Compare the proposed opt-out and selective-application approaches, then benchmark the amplicon-like command pattern to establish what behavior and speed improvement would count as done.

Written by the indexing model from the issue text.

Description

Hi - I was excited to see this upgrade as we use cutadapt extensively - it works great and can do things no other software of its kind can do, but is one of the slower components of our pipeline, so we're always interested in speed improvements.

However as it turns out we actually observed a mild decrease in speed (~10%) relative to earlier versions. Our use case is mainly trimming 5' adapters from "amplicon-like" data, whereby we load files of thousands of anchored adapter sequences (-g ^file:adapters1.fa -G ^file:adapters2.fa), and every read is expected to find a match to at least one sequence in the file (and possibly non-anchored matches to several). I'm not a software engineer but I wonder if for this use case the cost overhead of the new heuristic isn't worth it (even though most adapters won't be present in any particular read).

Would it be possible to either make this functionality optional via a switch, or else automatically turn it on selectively depending on the alignment mode and/or adapter characteristics? E.g. maybe it makes more sense for for non-anchored adapters vs. anchored (since adapters are more likely to be present in applications that involve anchored adapters, like primer sequences). Or perhaps when the adapters are too short relative to the error rate, the rate of coincidental matches in reads is too high; for a 20 nt adapter sequence with an error tolerance of 10% (2 errors), the kmers are so short that coincidental matches may not be rare.

The speed decrease isn't huge, and the new software version includes other useful features; it's just a bit of a bummer that at least for us the new heuristic is a cost not a benefit.

Originally posted by @eboyden in https://github.com/marcelm/cutadapt/issues/663#issuecomment-1483998923

Dominant language
Python
Stars
587
Forks
144
Avg merge
2h 37m
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from marcelm/cutadapt

All issues in marcelm/cutadapt

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.