CRAM incorrectly reporting success after malloc failure on read using threads
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
Start with test/test_view and the sam_read1() path, then inspect cram_eof(), cram_dopen(), cram_read_container(), and the error paths in cram/cram_decode.c. Reproduce the threaded malloc failure with the commands in the issue and trace where the EOF flag remains set. Done means the read reports failure and returns a nonzero status instead of success.
Written by the indexing model from the issue text.
Description
To reproduce, build HTSlib with @jkbonfield 's malloc testing framework. Create a small cram file:
FUZZ=1000000 ./test/test_view -C -t test/ce.fa test/ce#1000.sam > /tmp/ce#1000.cram
Try to read it back, but trigger a malloc failure. It will only print the header, log error messages but return success:
$ FUZZ=422 ./test/test_view -@ 2 -t test/ce.fa /tmp/ce#1000.cram
@SQ SN:CHROMOSOME_I LN:1009800 M5:8ede36131e0dbf3417807e48f77f3ebd UR:/nfs/users/nfs_r/rmd/git/htslib/test/ce.fa
@SQ SN:CHROMOSOME_II LN:5000 M5:8e7993f7a93158587ee897d7287948ec UR:/nfs/users/nfs_r/rmd/git/htslib/test/ce.fa
@SQ SN:CHROMOSOME_III LN:5000 M5:3adcb065e1cf74fafdbba1e8c352b323 UR:/nfs/users/nfs_r/rmd/git/htslib/test/ce.fa
@SQ SN:CHROMOSOME_IV LN:5000 M5:251af66a69ee589c9f3757340ec2de6f UR:/nfs/users/nfs_r/rmd/git/htslib/test/ce.fa
@SQ SN:CHROMOSOME_V LN:5000 M5:cf200a65fb754836dcc56b24b3170ee8 UR:/nfs/users/nfs_r/rmd/git/htslib/test/ce.fa
Malloc fail after 422 allocs at cram/cram_io.h:513
Malloc fail after 422 allocs at thread_pool.c:105
[E::cram_next_slice] Call to hts_tpool_next_result failed
$ echo $?
0
This is because sam_read1() calls cram_eof() to distinguish EOF from error, and in this case the cram eof flag is set. Note that following PR #91 the EOF flag is set in cram_dopen() and later cleared in cram_read_container(). This means there are places where failures can occur when it's set. A few of these set fd->eof = 0 before returning so the error gets reported, but it seems there are places where this isn't done.
- Dominant language
- C
- Stars
- 950
- Forks
- 475
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 11
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from samtools/htslib
-
Difficulty 3/5 1-2 days Newbie friendliness 74/100
-
Difficulty 3/5 1-2 days Newbie friendliness 76/100
-
tabix performance Open
Similar issues
-
[adam] AdamNet network read doesn't cap to MAX_ADAM_PACKET_LEN, overflows client receive buffers Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
FujiNetWIFI/fujinet-firmware#1649 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
HarbourMasters/Shipwright#7229 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
riscv-software-src/riscv-isa-sim#2435 · 1 comment ·
-
bug Self Built Image SNAPSHOT Supported Device target/ramips
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100