dbSNP 156 VCF now includes non-32 bit integers causing "Extreme INFO/RS value encountered and set to missing" errors
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Review the reported bcftools view commands and HTSlib's VCF_ALLOW_INT64 behavior first, then read the relevant BCF specification discussion in samtools/hts-specs. Done would require an agreed path for representing these values in BCF and a clearly scoped follow-up, since this issue asks whether the specification should change rather than naming an implementation target.
Written by the indexing model from the issue text.
Description
With release 156, now dbSNP includes rsIDs larger than 2^31 which cannot be properly handled by bcftools anymore:
$ wget https://ftp.ncbi.nih.gov/snp/redesign/latest_release/VCF/GCF_000001405.40.gz{,.tbi}
$ tabix GCF_000001405.40.gz NC_000001.11:6259533-6259533
NC_000001.11 6259533 rs2148352434 C T . . RS=2148352434;dbSNPBuildID=156;SSR=0;GENEINFO=GPR153:387509;VC=SNV;INT;R5;GNO;FREQ=1000Genomes:0.9998,0.0001562
$ bcftools view -H GCF_000001405.40.gz -r NC_000001.11:6259533-6259533
[W::vcf_parse_info] Extreme INFO/RS value encountered and set to missing at NC_000001.11:6259533
NC_000001.11 6259533 rs2148352434 C T . . RS=.;dbSNPBuildID=156;SSR=0;GENEINFO=GPR153:387509;VC=SNV;INT;R5;GNO;FREQ=1000Genomes:0.9998,0.0001562
If HTSlib is compiled with option -DVCF_ALLOW_INT64 then it works fine:
$ bcftools view -H GCF_000001405.40.gz -r NC_000001.11:6259533-6259533
NC_000001.11 6259533 rs2148352434 C T . . RS=2148352434;dbSNPBuildID=156;SSR=0;GENEINFO=GPR153:387509;VC=SNV;INT;R5;GNO;FREQ=1000Genomes:0.9998,0.0001562
However, this cannot be represented anymore as a binary VCF, which is a huge problem:
$ bcftools view -Ou GCF_000001405.40.gz -r NC_000001.11:6259533-6259533 | bcftools view -H
[E::bcf_write] Data at NC_000001.11:6259533 contains 64-bit values not representable in BCF. Please use VCF instead
[main_vcfview] Error: cannot write to (null)
Is there a discussion in samtools/hts-specs to get the BCF specification to update the specification to 64-bit values?
- Dominant language
- C
- Stars
- 891
- Forks
- 277
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from samtools/bcftools
-
needs-feedback
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
-
htslib-dependent P2: Desirable
Difficulty 3/5 1-2 days Newbie friendliness 48/100
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
-
needs-feedback requires-test-case
Difficulty 3/5 1-2 days Newbie friendliness 55/100
All issues in samtools/bcftools
Similar issues
-
[adam] AdamNet network read doesn't cap to MAX_ADAM_PACKET_LEN, overflows client receive buffers Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
FujiNetWIFI/fujinet-firmware#1649 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
HarbourMasters/Shipwright#7229 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
riscv-software-src/riscv-isa-sim#2435 · 1 comment ·
-
bug Self Built Image SNAPSHOT Supported Device target/ramips
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100