samtools index on uncompressed data

Open

#388 opened on Jun 9, 2016 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: c
Domain: data

Research direction

Start by reproducing the issue with an uncompressed BAM made from zcat foo.bam > bar.bam, then inspect the BGZF handling used by samtools index. Compare the resulting index behavior with samtools idxstats and regional retrieval. Done means indexing either rejects unsupported uncompressed data at creation time or supports retrieval correctly, according to the intended behavior.

Written by the indexing model from the issue text.

Description

Looking into the bgzf code I see that it supports reading of totally uncompressed data (eg from zcat foo.bam > bar.bam), apparently due to uncompressed bcf taking this form.

Curious as to how well this works, I tried samtools index on such a BAM file and it completed without complaint. However dies when attempting to use it:

[W::bam_hdr_read] EOF marker is absent. The input is probably truncated.
[main_samview] retrieval of region "2:100000000-100001000" failed due to truncated file or corrupt BAM index file

Samtools idxstats works though and gives identical output to the compressed BAM index.

It's a rather esoteric and therefore low priority case, but we should make the indexing code choke at creation time rather than usage time. (Unless it's actually meant to work and fakes up 64k uncompressed blocks just for the purpose of indexing and virtual offsets?)

Dominant language: C
Stars: 950
Forks: 475
Avg merge: 3d 13h
Merged PRs (30d): 11

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.