Pangenotype matrix: use a dense bit vector
- Dominant language
- Rust
- Stars
- 45
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Currently (as of #238), constructing a pangenotype matrix produces a `Vec>`. We could make this more memory-efficient by using a big bit vector. Fortunately, we already depend on [the bit-vec crate](https://docs.rs/bit-vec/latest/bit_vec/), so let's just use that.
This will require exposing a different interface to Python. We can consider using NumPy's [packbits](https://numpy.org/doc/stable/reference/generated/numpy.packbits.html) and its complement to translate back and forth between the dense format and usable matrices.
As part of this effort, we need to figure out how to benchmark the pangenotype matrix construction; I would like to know how much faster this optimization makes things go.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the pangenotype matrix construction introduced in #238 and trace its Rust representation through the Python interface. Review the existing bit-vec dependency and investigate NumPy packbits conversions, then establish a benchmark for construction so completion can be judged by a working dense representation, usable Python translation, and measured performance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, rust
- Domain
- data, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100