[DTLTO][COFF] clang-cl misreads the serialized remote compiler options: -O2 becomes -O3, -ffunction-sections/-fdata-sections/-floop-interchange are dropped
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
I noticed our compiler arguments weren't getting through when using DTLTO. The AI writeup is below.
---
For a COFF DTLTO link, `buildCommonRemoteCompilerOptions()` serializes the backend compiler invocation as:
`clang-cl.exe -c --target=x86_64-pc-windows-msvc19.51.36256 -O2 -faddrsig -ffunction-sections -fdata-sections -floop-interchange -Wno-unused-command-line-argument`
When the remote compiler is clang-cl, two of these options are misinterpreted and three are dropped, so the distributed backends do not reproduce the in-process ThinLTO backend.
### 1. Three of the serialized flags are not accepted by clang-cl
Every COFF DTLTO backend job prints:
clang-cl: warning: unknown argument ignored in clang-cl: '-ffunction-sections' [-Wunknown-argument]
clang-cl: warning: unknown argument ignored in clang-cl: '-fdata-sections' [-Wunknown-argument]
clang-cl: warning: unknown argument ignored in clang-cl: '-floop-interchange' [-Wunknown-argument]
The options are dropped. `lld/COFF/LTO.cpp` sets `FunctionSections`, `DataSections` and (by default) `PTO.LoopInterchange` for the in-process backend, and the serializer faithfully emits them, but in GCC spelling. The backend objects therefore come back with a single `.text` and a single `.data` section per module, and `/OPT:REF` and `/OPT:ICF` cannot operate at function or data granularity inside them.
### 2. `-O2` means `-O3` to clang-cl
`-O` is emitted as `-O2` for lld's default level. clang-cl treats `-O2` as `/O2`, and `TranslateOptArg` in `clang/lib/Driver/ToolChains/MSVC.cpp` expands `/O2`, `/Ox` and `/Ot` to cc1 `-O3`. So every clang-cl backend runs the O3 pipeline with aggressive codegen, while the in-process backend runs O2 (`LTOBackend.cpp` maps `Conf.OptLevel` 0-3 directly). Verified with `clang-cl -### -c -x ir`: `-O2`, `/O2`, `/Ox` and `/Ot` all produce cc1 `-O3`; only `/clang:-O2` (or `-Xclang -O2`) produces `-O2`. There is no cl-style `/O` flag that yields `-O2`.
Note that there is nothing to translate for size configurations: as of LLVM 23 there are no Os/Oz pipelines (#191363, #209424), and size intent is carried by the `optsize`/`minsize` attributes already in the bitcode, which the backend honors at any level.
### Measurements
Large application (~3,000 modules, 480,230 text functions), Win64, same bitcode objects and link flags, DTLTO vs in-process ThinLTO, clang/lld 23.1.0, attributing the difference per option via `/thinlto-remote-compiler-arg:`:
| remote options appended | `.text` | text functions | vs in-process |
|---|---|---|---|
| none (serialized options only) | 525.0 MB | 540,115 | 60,161 extra functions, no function granularity for /OPT:REF and /OPT:ICF |
| `/Gy /Gw` | 462.9 MB | 480,230 | function set matches, code 5.1% larger |
| `/Gy /Gw /clang:-O2` | 440.4 MB | 480,230 | `.text`, `.data`, `.pdata` byte-identical; all functions identical in size |
In-process: `.text` 440.4 MB, 480,230 functions. The same identity holds for `-Os` and for PGO instrumented and optimized configurations without any further option, confirming that size and profile intent travel in the IR. `-floop-interchange` being dropped is real but negligible on this code: disabling the pass in-process changes 3 functions and 224 bytes.
With the level and sections fixed, the one remaining difference is section order under `/call-graph-profile-sort` for PGO builds, because the sort breaks density ties on the order in which edges are read and the two paths deliver backend objects in different orders; with `/call-graph-profile-sort:no` the images are byte-identical.
### Suggestions
- When the remote compiler runs in cl driver mode (by name or by an explicit option), emit `/clang:-O` for the level and `/Gy`, `/Gw`, `/clang:-floop-interchange` (or `/clang:`-prefixed GCC spellings throughout) for the code generation options. `--thinlto-remote-compiler-prepend-arg` shows the plumbing already exists for driver-specific arguments.
- Treat a rejected translation option as an error rather than hiding it behind `-Wno-unused-command-line-argument`; today the rejection only surfaces because clang-cl reports it under `-Wunknown-argument`, and a build system that suppresses that warning loses the signal entirely.
Observed with clang/lld 23.1.0. Related: #220371.
Contributor guide
Research direction
Start by tracing option serialization in lld/COFF/LTO.cpp and the clang-cl translation behavior in clang/lib/Driver/ToolChains/MSVC.cpp. Use the documented DTLTO remote compiler invocation and --thinlto-remote-compiler-arg plumbing to verify that optimization, section, and loop-interchange options reach the backend with equivalent behavior to in-process ThinLTO.
Written by the indexing model from the issue text.
Assessment
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100