hardbyte / hardbyte/python-can
Optional use of ISA-L for zlib decode/encode operations
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 697
- PR merge metrics
- No merged PRs in 30d
Description
### Is your feature request related to a problem? Please describe.
I have an application that requires the decoding of very large .blf files, with hundreds of millions of can messages. I've modified the `BLFReader`class a lot, but I have observed that a very large speedup can be achieved very easily just by using a more optimized zlib decompression library, such as Intel's [ISA-L](https://github.com/intel/isa-l) (and the bindings provided by the [`isal`](https://pypi.org/project/isal/) python package) as a drop-in replacement.
### Describe the solution you'd like
The zlib import statement inside `can/io/blf.py` could be simply changed from:
```
import zlib
```
to:
```
try:
from isal import isal_zlib as zlib
except ImportError:
import zlib
```
This can be so that the library can be listed as optional in the installation instructions, and the remaining code can remain unmodified. I don't know about making it mandatory and/or replacing it altogether, as I haven't tested the `compress` routine (it isn't used in my specific application), but it should be fine.
Also I really haven't checked if the `zlib` module is used anywhere else, but obviously this change can be "universal".
### Additional context
This speedup works best when you need "skip" a lot of messages in the log (e.g. if you're only interested in can messages with a specific arbitration id and skip all of the others), and therefore need to sift trough this .blf file very quickly. I don't have any precise figures on the % speedup because it is very much context dependent, but it can be significant, especially on such very large files; also the package's installed size is around 540KB, so not too heavy.
Contributor guide
Assessment
This issue has not been assessed yet.