[clang-format] CRLF changes wrapping around a multiline raw string at ColumnLimit
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
### Summary
clang-format makes different wrapping and indentation decisions for LF and CRLF
copies of the same C++ fragment containing a multiline raw string. The difference
remains after normalizing output line endings.
This was noticed as inconsistent local and CI formatting. It reproduces using the
same Windows executable for both inputs.
### Versions and environment
- `clang-format version 20.1.7` and `clang-format version 22.1.5`.
- Windows 11, build 26200, x64.
- Binaries from the [clang-format Python distribution](https://github.com/ssciwr/clang-format-wheel),
installed in pre-commit environments.
- No Linux binary or build of LLVM main was tested for this report.
### Reproduction
Save the following as `repro.py` and run:
```text
python repro.py /path/to/clang-format
```
Only Python 3 and clang-format are needed. The script constructs both line-ending
variants explicitly and supplies the complete reduced style on the command line.
It does not depend on any files or ambient `.clang-format` configuration.
```python
import difflib
import subprocess
import sys
binary = sys.argv[1]
style = (
"{BasedOnStyle: WebKit, ColumnLimit: 104, "
"AllowAllArgumentsOnNextLine: false, BinPackArguments: false}"
)
source = b'''void f() {
document += alias
? fmt::format(
R"(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
)",
name,
a
)
: fmt::format(
R"({})",
name,
space_input
);
}
'''
print(subprocess.check_output([binary, "--version"]).decode().strip())
outputs = []
for data in [source, source.replace(b"\n", b"\r\n")]:
output = subprocess.check_output(
[binary, "--assume-filename=repro.cpp", "--style=" + style], input=data
)
outputs.append(output.replace(b"\r\n", b"\n").decode().splitlines(keepends=True))
print("".join(difflib.unified_diff(*outputs, fromfile="LF input", tofile="CRLF input")))
print("Identical formatting:", outputs[0] == outputs[1])
```
The embedded C++ is a formatting-only fragment; no application dependencies or
compilation are needed. The four style settings reproduce the discrepancy with a
104-column limit.
### Actual result
Both tested versions produce the following diff after output newline normalization:
```diff
--- LF input
+++ CRLF input
@@ -1,11 +1,12 @@
void f()
{
- document += alias ? fmt::format(
- R"(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ document += alias
+ ? fmt::format(
+ R"(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
)",
- name,
- a)
- : fmt::format(R"({})",
- name,
- space_input);
+ name,
+ a)
+ : fmt::format(R"({})",
+ name,
+ space_input);
}
```
The script prints `Identical formatting: False`.
### Expected result
The LF and CRLF inputs should produce the same wrapping and indentation after
normalizing output line endings: an empty diff and `Identical formatting: True`.
The LF layout above fits the configured column limit.
### Additional checks
The discrepancy also reproduces with `LineEnding: LF` or `LineEnding: CRLF` added
to the style, in both tested versions. Each output is stable on a second formatting
pass with the same configuration.
### Possible cause
In 20.1.7,
[`FormatTokenLexer::getNextToken()`](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.7/clang/lib/Format/FormatTokenLexer.cpp#L1373-L1392)
measures a multiline token's first line using the substring before the first LF,
which retains a preceding CR.
The [width helper](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.7/clang/lib/Format/Encoding.h#L42-L77)
falls back to byte length for nonprintable characters. This appears to count the CR
as an extra column when the LF layout's first raw-string line exactly meets the limit.
This explanation is based on source inspection; no patched LLVM build was tested.
### AI assistance
Assisted-by: OpenAI Codex.
AI assisted the investigation, reduction of the example, reproduction script, and
drafting of this report. The reported outputs were captured by executing the stated
formatter binaries.
Contributor guide
Research direction
Reproduce the difference with the provided repro.py and clang-format versions, then inspect FormatTokenLexer::getNextToken() in clang/lib/Format/FormatTokenLexer.cpp and the width helper in clang/lib/Format/Encoding.h. Add regression coverage for equivalent LF and CRLF inputs, and verify that normalized outputs match with the stated style and column limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100