bcftools reheader -f/--fai does not preserve the order of the contigs (even when it could)

Open
#1,425 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
c
Domain
cli

Research direction

Start by reproducing the issue with the provided ref.fai, chr1.bcf, and chr2.bcf commands, then compare the outputs of bcftools reheader and concat -n. Trace the reheader and concat command entry points to determine how contig order is handled. Done means files reheadered from the same index can be concatenated with -n without --naive-force and without changing record interpretation.

Written by the indexing model from the issue text.

Description

enhancement

I have noticed the following behavior.

Suppose I have two files and a reference index:

echo -e "##fileformat=VCFv4.2\n##contig=<ID=chr1,length=248956422>\n#CHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tINFO\nchr1\t1\t.\tA\tC\t.\t.\t." | bcftools view --no-version -Ob -o chr1.bcf
echo -e "##fileformat=VCFv4.2\n##contig=<ID=chr2,length=242193529>\n#CHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tINFO\nchr2\t1\t.\tG\tT\t.\t.\t." | bcftools view --no-version -Ob -o chr2.bcf
echo -e "chr1\t248956422\t112\t70\t71\nchr2\t242193529\t252513167\t70\t71" > ref.fai

I then use the reference index to reheader the VCFs:

bcftools reheader -f ref.fai -o chr1.reheader.bcf chr1.bcf
bcftools reheader -f ref.fai -o chr2.reheader.bcf chr2.bcf

Unintuitively, the order of the contigs is not the same in the two files now:

$ bcftools view --no-version chr1.reheader.bcf
##fileformat=VCFv4.2
##FILTER=<ID=PASS,Description="All filters passed">
##contig=<ID=chr1,length=248956422>
##contig=<ID=chr2,length=242193529>
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
chr1	1	.	A	C	.	.	.

$ bcftools view --no-version chr2.reheader.bcf
##fileformat=VCFv4.2
##FILTER=<ID=PASS,Description="All filters passed">
##contig=<ID=chr2,length=242193529>
##contig=<ID=chr1,length=248956422>
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
chr2	1	.	G	T	.	.	.

This makes it impossible to concatenate files with the -n option:

$ bcftools concat -n chr1.reheader.bcf chr2.reheader.bcf
Checking the headers of 2 files.
Cannot use --naive, use --naive-force instead: different order the tag contig/chr1 in chr1.reheader.bcf vs chr2.reheader.bcf

And, of course, using the --naive-force option would cause inappropriate behavior:

$ bcftools concat -n --naive-force chr1.reheader.bcf chr2.reheader.bcf | bcftools view --no-version
Concatenating chr1.reheader.bcf	0.000709 seconds
Concatenating chr2.reheader.bcf	0.000503 seconds
##fileformat=VCFv4.2
##FILTER=<ID=PASS,Description="All filters passed">
##contig=<ID=chr1,length=248956422>
##contig=<ID=chr2,length=242193529>
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
chr1	1	.	A	C	.	.	.
chr1	1	.	G	T	.	.	.

However, I do understand why bcftools reheader -f would not want to reorder the contigs, as that would potentially cause the VCF file to become unsorted. At the same time I thought that the bcftools concat -n option was exactly supposed to work after the header of the VCF files were updated with the bcftools reheader -f option.

Dominant language
C
Stars
891
Forks
277
Avg merge
2d 23h
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from samtools/bcftools

All issues in samtools/bcftools

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.