Profile cmake generation
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
I recently learned you can profile cmake using `--profiling-format=google-trace --profiling-output=cmake_profile.json`. Since running cmake is not fast, I wondered what we can find. CMake configure time matters especially on clusters with network file systems, were cmake generation can take a couple of minutes.
So I tried it using a cmake command I commonly use to get the `all-dev` preset:
```bash
cccl/build_prof$ CXX=clang++-20 cmake .. --preset all-dev -B. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_CUDA_HOST_COMPILER=clang++-20 -DTHRUST_ENABLE_MULTICONFIG=OFF -DCCCL_ENABLE_BENCHMARKS=OFF -DCMAKE_CUDA_FLAGS="--ftemplate-backtrace-limit=0 -lineinfo" -DCMAKE_CUDA_ARCHITECTURES=120 --profiling-format=google-trace --profiling-output=cmake_profile.json
```
results in this graph:
We can see that fetching dependencies via `fetchcontent` is expensive.
On a second run of the above command, where we already have an existing binary directory with fetched dependencies:
We now see that generating the build system takes the most time. I assume because we have a LOT of targets.
This is for CUB tuning, which we run very frequently on clusters:
```
cccl/build_prof$ cmake .. --preset=cub-tune
```
We can see that `try_compile` from CMake takes some time (I guess we cannot change that), and the other large block is getting nvbench, which pulls in catch2 (which we wouldn't need as consumers of nvbench), and libfmt and nlohmann_json (both via rapids_cpm_find).
We could consider not fetching Catch2 when nvbench is consumed as sub project, and refactoring libfmt to `std::format` so we can lose the dependency.
In any case, profiling cmake is surprisingly easy and insightful and there are probably some low hanging fruits on how to configure faster that we could try.
Contributor guide
Assessment
This issue has not been assessed yet.