feature request: duplicate merging and selecting

Open
#2,342 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
c
Domain
bioinformatics, cli

Research direction

Start with the norm duplicate-handling path, especially --rm-dup, and compare it with merge's --info-rules behavior. Define which duplicate selection, merging, or marking behavior is in scope and how it should be verified against the gnomAD coordinate-lifting use case; the issue currently presents several alternatives rather than one settled outcome.

Written by the indexing model from the issue text.

Description

My use case is lifting over gnomAD v4.1 from GRCh38 to T2T-CHM13v2.0 and sometimes multiple GRCh38 variants resolve to the same T2T coordinate - I want to be able to process these duplicates (say picking highest or lowest AF) rather than just taking the first in the file

Control how selecting duplicates works

It would be really useful to be able to choose which one to take. You could do this by defining how to sort the dupes then taking the 1st, for instance take the one with the highest AF, then highest AC with --rm-dup-sort=-AF,-AC or --rm-dup--sort=AF:desc,AC:desc

Merge functionality with duplicates

Merge has --info-rules which works with the same variant across different files. It would be nice to be able to apply this to same variant in the same file, for instance norm --rm-dup --info-rules=BCFTOOLS_OLD_VARIANT:join would have allowed a workaround for this issue

Mark duplicates

Another way to solve this would be to mark duplicates rather than remove them, for instance a DUPLICATE flag.

Then I could select them out into a separate file and:

  • Use existing merge to bring them back with --info-rules
  • Process this much smaller file in Python and process them however I want then merge back (much quicker than processing ~100G of compressed VCF in Python)
Dominant language
C
Stars
891
Forks
277
Avg merge
2d 23h
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from samtools/bcftools

All issues in samtools/bcftools

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.