investigate using precompiled headers to accelerate compilation
- Dominant language
- C++
- Stars
- 245
- Forks
- 36
- Avg merge
- 8d 23h
- Merged PRs (30d)
- 4
Description
Profiling the compilation of sourcefiles in serac reveals a considerable amount of time is being spent parsing headers.
Some of the more expensive ones include (no pun intended):
```cpp
#include // ~ 300 ms
#include "mfem.hpp" // ~1250 ms (!)
#include "axom/inlet.hpp" // ~2200 ms (!!)
#include "functional.hpp" // ~ 150 ms
```
The values above were taken from a profile of compiling `functional_comparisons.cpp` with clang-10.0.0 (raw data available [here](https://github.com/LLNL/serac/files/7214646/functional_comparisons_profiling.zip), view flamegraph with `chrome://tracing`). These numbers represent the time to merely parse the header, even if it is unused in the rest of the translation unit (note: the times are even worse when compiling a sourcefile with NVCC, as each file is processed more than once). These headers are present in many of our translation units, meaning that these compilation costs are incurred dozens of times within serac.
CMake 3.16+ supports defining precompiled headers that help to avoid paying this parsing cost in every translation unit. This small project [here](https://github.com/LLNL/serac/files/7214620/pch_test.zip) demonstrates the potential gain, using mfem's monolithic `mfem.hpp` header as an example:
without using precompiled headers:
```
sam@provolone:~/code/pch_test/build$ touch ../*.cpp && time make
Consolidate compiler generated dependencies of target mfem
[ 95%] Built target mfem
Consolidate compiler generated dependencies of target pch
[ 97%] Building CXX object CMakeFiles/pch.dir/foo.cpp.o
[ 97%] Building CXX object CMakeFiles/pch.dir/bar.cpp.o
[ 97%] Building CXX object CMakeFiles/pch.dir/baz.cpp.o
[ 97%] Building CXX object CMakeFiles/pch.dir/qux.cpp.o
[100%] Building CXX object CMakeFiles/pch.dir/main.cpp.o
[100%] Linking CXX executable pch
[100%] Built target pch
real 0m3.917s
user 0m3.537s
sys 0m0.374s
```
and after precompiling `mfem.hpp`:
```
sam@provolone:~/code/pch_test/build$ touch ../*.cpp && time make
Consolidate compiler generated dependencies of target mfem
[ 95%] Built target mfem
Consolidate compiler generated dependencies of target pch
[ 97%] Building CXX object CMakeFiles/pch.dir/foo.cpp.o
[ 97%] Building CXX object CMakeFiles/pch.dir/bar.cpp.o
[ 97%] Building CXX object CMakeFiles/pch.dir/baz.cpp.o
[100%] Building CXX object CMakeFiles/pch.dir/qux.cpp.o
[100%] Building CXX object CMakeFiles/pch.dir/main.cpp.o
[100%] Linking CXX executable pch
[100%] Built target pch
real 0m1.480s
user 0m1.138s
sys 0m0.325s
```
Each of the sourcefiles `foo.cpp`, `bar.cpp`, `baz.cpp`, `qux.cpp` contain essentially nothing other than including the unused header file to focus the timings on the `#include "mfem.hpp"` statement. e.g.
```
sam@provolone:~/code/pch_test$ cat foo.cpp
#include "mfem.hpp"
double foo() { return 42.0; }
```
In this little proof of concept, there was a significant savings in compilation time (~3x speedup). It would be great if serac as a whole could benefit from this approach as well.
Contributor guide
Research direction
Start by reviewing the CMake configuration and the functional_comparisons.cpp profiling data, then reproduce the compilation timings described in the issue. Evaluate precompiled headers for the repeatedly included mfem.hpp and axom/inlet.hpp headers, including the NVCC build path. Done means the project builds correctly and demonstrates a measured compilation-time improvement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100