cube-js / cube-js/cube

Evaluate using Profile-Guided Optimization (PGO) and LLVM BOLT

Open
#7,354 1 comment 2 reactions 0 assignees View on GitHub
help wanted
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

**Is your feature request related to a problem? Please describe.**
Not a problem - just an idea of how to potentially improve the Cube's performance.

**Describe the solution you'd like**

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are available [here](https://github.com/zamazan4ik/awesome-pgo/). According to the tests, PGO helps with achieving better performance in many software domains like databases, compilers, network applications, etc. I think trying to optimize Cube (its Rust part since Rust supports PGO) with PGO can be a good idea.

I can suggest the following action points:
* Perform PGO benchmarks on Cube. And if it shows improvements - add a note about possible improvements in Cube's performance with PGO.
* Providing an easier way (e.g. a build option) to build scripts with PGO can be helpful for the end-users and maintainers since they will be able to optimize Cube according to their own workloads.
* Optimize pre-built Cube binaries (like Docker containers) (if it's possible to prepare a good enough workload for PGO training)

Testing Post-Link Optimization techniques (like [LLVM BOLT](https://github.com/llvm/llvm-project/blob/main/bolt/README.md)) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

**Additional context**
For the Rust projects, I recommend starting with [cargo-pgo](https://github.com/Kobzol/cargo-pgo). More details about PGO support in Rust can be found in the [official docs](https://doc.rust-lang.org/rustc/profile-guided-optimization.html).

Here are some examples of how PGO optimization is integrated in other projects:

* Rustc: a CI [script](https://github.com/rust-lang/rust/blob/master/src/ci/stage-build.py) for the multi-stage build
* GCC:
- Official [docs](https://gcc.gnu.org/install/build.html), section "Building with profile feedback" (even AutoFDO build is supported)
- A [part](https://github.com/gcc-mirror/gcc/blob/4832767db7897be6fb5cbc44f079482c90cb95a6/configure#L7818) in a "wonderful" `configure` script
* Clang: [Docs](https://llvm.org/docs/HowToBuildWithPGO.html)
* Python:
- CPython: [README](https://github.com/python/cpython#profile-guided-optimization)
- Pyston: [README](https://github.com/pyston/pyston#building)
* Go: [Bash script](https://github.com/golang/go/blob/master/src/cmd/compile/profile.sh)
* V8: [Bazel flag](https://github.com/v8/v8/blob/main/BUILD.gn#L184)
* ChakraCore: [Scripts](https://github.com/chakra-core/ChakraCore/tree/master/Build/scripts/pgo)
* Chromium: [Script](https://chromium.googlesource.com/chromium/src/build/config/+/refs/heads/main/compiler/pgo/BUILD.gn)
* Firefox: [Docs](https://firefox-source-docs.mozilla.org/build/buildsystem/pgo.html)
- Thunderbird has PGO support too
* PHP - [Makefile command](https://github.com/php/php-src/blob/master/build/Makefile.global#L138) and old Centminmod [scripts](https://github.com/centminmod/php_pgo_training_scripts)
* MySQL: [CMake script](https://github.com/mysql/mysql-server/blob/8.0/cmake/fprofile.cmake)
* YugabyteDB: [GitHub commit](https://github.com/yugabyte/yugabyte-db/commit/34cb791ed9d3d5f8ae9a9b9e9181a46485e1981d)
* FoundationDB: [Script](https://github.com/apple/foundationdb/blob/1a6114a66f3de508c0cf0a45f72f3687ba05750c/contrib/generate_profile.sh)
* Zstd: [Makefile](https://github.com/facebook/zstd/blob/dev/programs/Makefile#L232)
* [Foot](https://codeberg.org/dnkl/foot): [Scripts](https://codeberg.org/dnkl/foot/src/branch/master/pgo)
* Windows Terminal: [GitHub PR](https://github.com/microsoft/terminal/pull/10071)
* Pydantic-core: [GitHub PR](https://github.com/pydantic/pydantic-core/pull/741)
* file.d: [GitHub PR](https://github.com/ozontech/file.d/pull/469)
* OceanBase: [CMake flag](https://github.com/oceanbase/oceanbase/blob/master/cmake/Env.cmake#L55)
* NodeJS: [Configure script](https://github.com/nodejs/node/commit/9be15559cc0bfe506d9cdfba4ad0f4beacf5ce17)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.