Performance : slowness during 'packing' time with Hermit CLI.
- Dominant language
- C
- Stars
- 230
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
I am opening this issue to discuss possible performance improvements that can be brought to Hermit CLI.
Measurements below are based on PR #23 : the throughput of `hermit.com` is not constant and degrades with medium to big inputs, the time required for packing increases with larger inputs. few data points below :
| | Count Vowels| Ubuntu | Emacs |
|--------|--------|--------|--------|
| Pack time|795ms|1m33s|8m30s|
| Throughput |2.6Mb/s|1.7Mb/s|1.6Mb/s|
| .wasm size |2.1Mb|160Mb|853Mb|
| .com size |1.2Mb|63Mb|330Mb|
*Wasm files of Ubuntu and Emacs were generated using [container2wasm](https://github.com/ktock/container2wasm).
After few runs with [verbose mode](https://github.com/mtb0x1/hermit/commit/b872a8184003ab13a1ccf073b2cbfca10e6810dd), a pattern emerged. The root cause of the slowness is the zipping of `*.wasm` in https://github.com/dylibso/hermit/blob/main/hermit-cli/crates/hermitfile-parser/src/main.rs#L181-L187.
A change of config as follow, results in a different performance results:
https://github.com/mtb0x1/hermit/blob/zip_stored/hermit-cli/crates/hermitfile-parser/src/main.rs#L198-L214
| | Count Vowels| Ubuntu | Emacs |
|--------|--------|--------|--------|
| Pack time|39ms|0m1s30ms|0m7s80ms|
| Throughput |53Mb/s|123Mb/s|109Mb/s|
| .wasm size |2.1Mb|160Mb|853Mb|
| .com size |2.7M|161Mb|853Mb|
A drastic change in performance that have a downside, the size of produced artifacts. but in the other hand, I don't think that users will expect any 'out of the box' binaries size reduction.
My suggestion would be to add options to Hermit cli to allow the user to choose a 'packing' strategy.
the strategy would be somehow similar to what classic compilers have already :
- Optimization for size : would allow a level of compression to be [specified](https://docs.rs/zip/latest/zip/write/struct.FileOptions.html#method.compression_level).
- Optimization for speed (use [Stored](https://docs.rs/zip/latest/zip/enum.CompressionMethod.html#variant.Stored) instead of [Deflated](https://docs.rs/zip/latest/zip/enum.CompressionMethod.html#variant.Deflated))
N.B :
- Modifications introduced have an impact on both `hermit.com` and artifacts produced by it. I would recommend separating this in 2 stages and avoid sharing same optimization strategy.
- Modifications introduced at the 'packing' step have an impact on runtime of artifacts. (faster boot of .com CLI, still slow but this would be a subject of another issue in the future.)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in hermit-cli/crates/hermitfile-parser/src/main.rs, especially the packing code around lines 181-187 and the configuration change shown around lines 198-214. Review the reported measurements and the zip crate's compression options. Done means Hermit CLI supports selectable size- or speed-oriented packing strategies while accounting for the stated effects on hermit.com and produced artifacts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, wasm
- Domain
- cli, compilers, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100