hardbyte / hardbyte/python-can
Optional use of ISA-L for zlib decode/encode operations
- Ngôn ngữ chính
- Python
- Star
- 1.6k
- Fork
- 697
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
### Is your feature request related to a problem? Please describe.
I have an application that requires the decoding of very large .blf files, with hundreds of millions of can messages. I've modified the `BLFReader`class a lot, but I have observed that a very large speedup can be achieved very easily just by using a more optimized zlib decompression library, such as Intel's [ISA-L](https://github.com/intel/isa-l) (and the bindings provided by the [`isal`](https://pypi.org/project/isal/) python package) as a drop-in replacement.
### Describe the solution you'd like
The zlib import statement inside `can/io/blf.py` could be simply changed from:
```
import zlib
```
to:
```
try:
from isal import isal_zlib as zlib
except ImportError:
import zlib
```
This can be so that the library can be listed as optional in the installation instructions, and the remaining code can remain unmodified. I don't know about making it mandatory and/or replacing it altogether, as I haven't tested the `compress` routine (it isn't used in my specific application), but it should be fine.
Also I really haven't checked if the `zlib` module is used anywhere else, but obviously this change can be "universal".
### Additional context
This speedup works best when you need "skip" a lot of messages in the log (e.g. if you're only interested in can messages with a specific arbitration id and skip all of the others), and therefore need to sift trough this .blf file very quickly. I don't have any precise figures on the % speedup because it is very much context dependent, but it can be significant, especially on such very large files; also the package's installed size is around 540KB, so not too heavy.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.