ngless-toolkit / ngless-toolkit/ngless

Reading .bz2 files fails to decompress or segfaults

Open
#116 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
152
Forks
23
PR merge metrics
No merged PRs in 30d

Description

This was tested using the 1.0.0 conda build (is this one just the wrapped static build?) as well as with several different 'static' and containerized versions from 0.9 to 1.0.1.

In all cases loading of data failed at the same step but depending on the version and how it was compiled two errors were seen:

...
[Wed 31-07-2019 11:24:23] Line 13: Created & opened temporary file /tmp/preprocessed.singles...fq12609-4.gz
/.singularity.d/runscript: line 3: 12609 Segmentation fault      (core dumped) ngless "$@"

and

...
[Wed 31-07-2019 11:23:51] Line 13: Created & opened temporary file /tmp/preprocessed.singles...fq8945-4.gz
Exiting after internal error. If you can reproduce this issue, please run your script with the --trace flag and report a bug at http://github.com/ngless-toolkit/ngless/issues
user error (BZ2_bzDecompress: -1)

We didn't try the docker containers but those also make use of the static builds so they should be equally affected.

I also tried using the same binary on the bz2 files in our testsuite and all worked fine which hints at some buffer or filesize related issue.

Currently in the process of creating a bz2 file that is big enough to trigger the error locally. If not too big I'll add this to the testsuite.

Credits to @jakob-wirbel for finding this bug.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the affected .bz2 input and the testsuite data, using the --trace flag as requested in the issue. Compare behavior across the static and conda builds, then add a sufficiently large reproducer to the testsuite if feasible; done means large .bz2 files decompress without an internal error or segmentation fault.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
bioinformatics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.