aonez / aonez/Keka

Some more performance testing for bzip2 and gzip

未关闭
#157 4 条评论 0 个 reaction 已指派 1 人 已被 @aonez 认领 在 GitHub 查看
blessed bzip2 gzip performance tests
主要语言
PHP
星标
7.3k
派生
294
平均合并
14 天 16 小时
30 天内合并 PR
1

描述

I did some testing… comparing 7z to lbzip2 and pbzip2…

Both pbzip2 and lbzip2 seem super fast, much faster than 7z regardless of the compression level you choose for 7z… even 7z compressing bzip2 on level 1 (which should be it’s fastest) took like 25 seconds and produced a 345.2MB file (original file was about 730MB)...

lbzip2 compressed the same file to 341.1MB in about 10 seconds…
lbzip2 -z -9 -n 10

pbzip2 compressed a file to 341MB (slightly more compressed) in a similar time
pbzip2 -z -9

Using lower levels with pbzip2 & lbzip2 didn’t seem to make much sense in my opinion because even on level 1 pbzip2 and lbzip2 seemed like they were about as fast as on level 9, but the file was bigger (345.5MB for pbzip2 and 346.2MB for lbzip2)… so just use level 9 if it’s super-optimized for that level... I'm not sure would pbzip2 & lbzip2 compressing on lower levels be noticeably faster on some much older Mac, but they seemed pretty fast even in Parallels with limited number of CPU cores they can use.

On level 4 7z compressed the file to 340.6MB with bzip2 format (so, slightly better compressed than pbzip2 did on lelev 9 in about 10 seconds), but it needed about 25 seconds for that… so I don’t think it makes much sense to use 7z on level 4 for bzip2 if it’s only slightly better compressed than pbzip2 on level 9, but takes about 25 seconds instead of about 10… for higher 7z levels, where the difference in filesize would be bigger, I guess it would make sense to use 7z then.

So I think that on “normal” level you should go with pbzip2… and since it’s really fast, I think that you don’t need any lower levels for bzip2, even for older machines because how super-optimized pbzip2 seems to be… “normal compression (fast)” could be default and if someone wants highest compression regardless of speed, you could have an option “high compression (very slow)” with 7z on maybe level 7 and “extreme (ultra sow)” with 7z on it's level 9.

BTW... right now, in latest Keka beta, Keka seems to be using 7z to compress to bzip2 if you are only compressing one file, but it uses lbzip2 if you drag & drop multiple files, so compressing one file to bzip2 with Keka is slower than compressing that same file and a bunch of some other files together.

There seems to be a similar issue when compressing a single file to gzip with Keka too... Keka only seems to use pigz for compression if you compress multiple files and pigz also seems super-fast... but when compressing a single file, it uses 7z to compress to gzip and that just feels super-slow when compared to pigz compression...

7z doesn't even seem to be using multithreading when compressing to gzip...

I'll do some more testing with gzip format to see could 7z on it's highest settings compress to gzip significantly more than pigz to justify using 7z to compress to gzip on slow settings without multithreading... and maybe how would it compare to zopfli... maybe it could be used just on slowest settings as some "ultra slow" option and when using pigz compression call it "super fast".

Also... Don't forget to use pigz and pbzip2 for decompressing bzip2 and gzip files, for better performance.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。