facebook / facebook/rocksdb

cmake support for link-time optimization

Open
#7,186 1 comment 0 reactions 0 assignees View on GitHub
up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

Among other things, link-time optimizations enable more devirtualization opportunities and inlining across translation units. This can save significant overhead introduced by RocksDB's pluggable interfaces/internal abstractions, like in the iterator hierarchy. https://github.com/facebook/rocksdb/pull/7181 added the corresponding support for Makefile.

With cmake we want to be able to enable LTO successfully just by setting `-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=True`. Currently there are some barriers:

- `-DCMAKE_POLICY_DEFAULT_CMP0069=NEW` is needed. It could be elided if we bump the value in `cmake_minimum_required()` to 3.9.
- The build ultimately fails as shown below. We should test it with different compilers/build modes and verify it works.

```
$ cmake .. -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=True -DCMAKE_POLICY_DEFAULT_CMP0069=NEW -DCMAKE_BUILD_TYPE=Release
$ make -j48 db_bench
...
[ 92%] Built target rocksdb
/data/users/andrewkr/rocksdb/table/block_based/block_based_table_builder.cc: In member function ‘EnterUnbuffered’:
/usr/include/c++/4.8.2/bits/unique_ptr.h:234:0: error: ‘block_rep’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
{ return std::get<0>(_M_t); }
^
/data/users/andrewkr/rocksdb/table/block_based/block_based_table_builder.cc:1598:41: note: ‘block_rep’ was declared here
ParallelCompressionRep::BlockRep* block_rep;
^
lto1: all warnings being treated as errors
lto-wrapper: /bin/c++ returned 1 exit status
/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [librocksdb.so.6.11.0] Error 1
make[2]: Leaving directory `/data/users/andrewkr/rocksdb/cmake-bld'
make[1]: *** [CMakeFiles/rocksdb-shared.dir/all] Error 2
make[1]: Leaving directory `/data/users/andrewkr/rocksdb/cmake-bld'
make: *** [all] Error 2
```

Built another way, the error looks like:

```
$ cmake .. -DCMAKE_CXX_COMPILER=clang++.par -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=True -DCMAKE_POLICY_DEFAULT_CMP0069=NEW -DCMAKE_BUILD_TYPE=Release
$ make -j48 VERBOSE=1
...
/mnt/gvfs/third-party2/binutils/08634589372fa5f237bfd374e8c644a8364e78c1/2.29.1/centos7-native/da39a3e/bin/ld: /mnt/gvfs/third-party2/llvm-fb/7318eaac22659b6ff2fe43918e4b69fd0772a8a7/20181009/platform007/9e92d53/bin/../lib/LLVMgold.so: error loading plugin: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /mnt/gvfs/third-party2/llvm-fb/7318eaac22659b6ff2fe43918e4b69fd0772a8a7/20181009/platform007/9e92d53/bin/../lib/LLVMgold.so)
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/rocksdb-shared.dir/build.make:3972: librocksdb.so.6.11.0] Error 1
make[2]: Leaving directory '/data/users/andrewkr/rocksdb/cmake-bld'
make[1]: *** [CMakeFiles/Makefile2:248: CMakeFiles/rocksdb-shared.dir/all] Error 2
make[1]: Leaving directory '/data/users/andrewkr/rocksdb/cmake-bld'
make: *** [Makefile:130: all] Error 2
```

As a followup task, we should also figure out profile-guided optimization support in cmake/Makefile.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.