Add ISA-L to accelerate gzip compression in case of high network load
- Dominant language
- C++
- Stars
- 28.9k
- Forks
- 5.6k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 430
Description
*Title*: *Add ISA-L to accelerate gzip compression in case of high network load*
*Description*:
[ISA-L](https://github.com/intel/isa-l) is a collection of optimized low-level functions targeting storage applications. The low-level functions of ISA-L are written in manual assembly code that many details are tuned to achieve optimal performance.
ISA-L uses efficient SIMD (Single Instruction, Multiple Data) instructions like AVX512 to maximize the use of CPU micro-architecture to accelerate the calculation process of compression algorithms. And ISA-L mainly optimized the implementation of CRC, DEFLATE and Huffman encoding for gzip compression algorithm.
We have performed a benchmark that test the compression performance under different data scales with ISA-L and Zlib, see figure in bellow. Specifically, we used [compressor_speed_test tool](https://github.com/envoyproxy/envoy/blob/main/test/extensions/filters/http/compressor/compressor_filter_speed_test.cc) to do benchmark, and averaged results of all compression levels for drawing the graph. And you can see all benchmark logs in [isa-l-benchmark](https://github.com/giantcroc/isa-l-benchmark).

As shown in the figure above, we believe that ISA-L can help Envoy to improve gzip compression performance in case of **high network load**, and we are working on integrating ISA-L into Envoy.
[optional *Relevant Links*:]
[ISA-L Igzip: Improvements to a Fast Deflate](https://www.computer.org/csdl/proceedings-article/dcc/2017/07923748/12OmNyugyUz)
[ISA-L tool](https://www.intel.com/content/www/us/en/developer/tools/isa-l/overview.html)
[ISA-L videos](https://www.intel.cn/content/www/cn/zh/developer/videos/intelligent-storage-acceleration-library.html)
[ISA-L intro](https://s3.us-east-2.amazonaws.com/intel-builders/day_2_intelligent_storage_acceleration_library.pdf)
[ISA-L in DPDK](https://doc.dpdk.org/guides/compressdevs/isal.html)
Contributor guide
Assessment
This issue has not been assessed yet.