cloudflare / cloudflare/lol-html

Evaluate using Profile-Guided Optimization (PGO) and Post-Link Optimization (PLO)

Open
#202 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2.1k
Forks
111
PR merge metrics
No merged PRs in 30d

Description

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are available [here](https://github.com/zamazan4ik/awesome-pgo/). According to the tests, PGO can help with achieving better performance in many cases similar to `lol-html`. I think trying to optimize `lol-html` with PGO can be a good idea.

I already did some benchmarks and want to share my results.

## Test environment

* Fedora 39
* Linux kernel 6.5.12
* AMD Ryzen 9 5900x
* 48 Gib RAM
* SSD Samsung 980 Pro 2 Tib
* Compiler - Rustc 1.74
* lol-html version: the latest for now from the `master` branch on commit `44a7659d1ce018c27ae1f7e7913884bdedf72d71`
* Disabled Turbo boost

## Benchmark

For benchmark purposes, I use `cargo bench` benchmark. For PGO optimization I use [cargo-pgo](https://github.com/Kobzol/cargo-pgo) tool. The same benchmark suite was used for the PGO training phase built with `cargo pgo bench`. PGO optimized results I got with `cargo pgo optimize bench`.

## Results

I got the following results:

* PGO optimized compared to Release: https://gist.github.com/zamazan4ik/e838bdc873d026bab9fbe21bfed99a8e
* (just for reference) PGO instrumentation compared to Release: https://gist.github.com/zamazan4ik/c94afd268a1ad39d8f8bbe3f17cf1109

As I interpret the results, PGO measurably improves `lol-html` performance in many cases. Since this library is used internally in Cloudflare Workers, such performance improvement can be valuable.

## Further steps

I can suggest the following action points:

* Perform more PGO benchmarks on `lol-html`. If it shows improvements - add a note to the documentation about possible improvements in `lol-html` performance with PGO.
* Providing an easier way (e.g. a build option) to build scripts with PGO can be helpful for the end-users and maintainers since they will be able to optimize `lol-html` according to their workloads.

Testing Post-Link Optimization techniques (like [LLVM BOLT](https://github.com/llvm/llvm-project/blob/main/bolt/README.md)) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

Here are some examples of how PGO optimization is integrated into other projects:

* Rustc: a CI [script](https://github.com/rust-lang/rust/blob/master/src/ci/stage-build.py) for the multi-stage build
* GCC:
- Official [docs](https://gcc.gnu.org/install/build.html), section "Building with profile feedback" (even AutoFDO build is supported)
- A [part](https://github.com/gcc-mirror/gcc/blob/4832767db7897be6fb5cbc44f079482c90cb95a6/configure#L7818) in a "wonderful" `configure` script
* Clang: [Docs](https://llvm.org/docs/HowToBuildWithPGO.html)
* Python:
- CPython: [README](https://github.com/python/cpython#profile-guided-optimization)
- Pyston: [README](https://github.com/pyston/pyston#building)
* Go: [Bash script](https://github.com/golang/go/blob/master/src/cmd/compile/profile.sh)
* V8: [Bazel flag](https://github.com/v8/v8/blob/main/BUILD.gn#L184)
* ChakraCore: [Scripts](https://github.com/chakra-core/ChakraCore/tree/master/Build/scripts/pgo)
* Chromium: [Script](https://chromium.googlesource.com/chromium/src/build/config/+/refs/heads/main/compiler/pgo/BUILD.gn)
* Firefox: [Docs](https://firefox-source-docs.mozilla.org/build/buildsystem/pgo.html)
- Thunderbird has PGO support too
* PHP - [Makefile command](https://github.com/php/php-src/blob/master/build/Makefile.global#L138) and old Centminmod [scripts](https://github.com/centminmod/php_pgo_training_scripts)
* MySQL: [CMake script](https://github.com/mysql/mysql-server/blob/8.0/cmake/fprofile.cmake)
* YugabyteDB: [GitHub commit](https://github.com/yugabyte/yugabyte-db/commit/34cb791ed9d3d5f8ae9a9b9e9181a46485e1981d)
* FoundationDB: [Script](https://github.com/apple/foundationdb/blob/1a6114a66f3de508c0cf0a45f72f3687ba05750c/contrib/generate_profile.sh)
* Zstd: [Makefile](https://github.com/facebook/zstd/blob/dev/programs/Makefile#L232)
* [Foot](https://codeberg.org/dnkl/foot): [Scripts](https://codeberg.org/dnkl/foot/src/branch/master/pgo)
* Windows Terminal: [GitHub PR](https://github.com/microsoft/terminal/pull/10071)
* Pydantic-core: [GitHub PR](https://github.com/pydantic/pydantic-core/pull/741)
* file.d: [GitHub PR](https://github.com/ozontech/file.d/pull/469)
* OceanBase: [CMake flag](https://github.com/oceanbase/oceanbase/blob/master/cmake/Env.cmake#L55)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the existing `cargo bench` results, then compare the training and optimized runs described with `cargo pgo bench` and `cargo pgo optimize bench`. Review the project documentation and build entry points to determine whether the work should document PGO or provide a build option. Done requires agreed benchmark evidence and a clearly scoped documentation or integration change.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.