llvm / llvm/llvm-project

[clang-format] CRLF changes wrapping around a multiline raw string at ColumnLimit

Open
#224,571 1 comment 0 reactions 0 assignees View on GitHub
clang-format
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

### Summary

clang-format makes different wrapping and indentation decisions for LF and CRLF
copies of the same C++ fragment containing a multiline raw string. The difference
remains after normalizing output line endings.

This was noticed as inconsistent local and CI formatting. It reproduces using the
same Windows executable for both inputs.

### Versions and environment

- `clang-format version 20.1.7` and `clang-format version 22.1.5`.
- Windows 11, build 26200, x64.
- Binaries from the [clang-format Python distribution](https://github.com/ssciwr/clang-format-wheel),
installed in pre-commit environments.
- No Linux binary or build of LLVM main was tested for this report.

### Reproduction

Save the following as `repro.py` and run:

```text
python repro.py /path/to/clang-format
```

Only Python 3 and clang-format are needed. The script constructs both line-ending
variants explicitly and supplies the complete reduced style on the command line.
It does not depend on any files or ambient `.clang-format` configuration.

```python
import difflib
import subprocess
import sys

binary = sys.argv[1]
style = (
"{BasedOnStyle: WebKit, ColumnLimit: 104, "
"AllowAllArgumentsOnNextLine: false, BinPackArguments: false}"
)
source = b'''void f() {
document += alias
? fmt::format(
R"(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
)",
name,
a
)
: fmt::format(
R"({})",
name,
space_input
);
}
'''
print(subprocess.check_output([binary, "--version"]).decode().strip())
outputs = []
for data in [source, source.replace(b"\n", b"\r\n")]:
output = subprocess.check_output(
[binary, "--assume-filename=repro.cpp", "--style=" + style], input=data
)
outputs.append(output.replace(b"\r\n", b"\n").decode().splitlines(keepends=True))
print("".join(difflib.unified_diff(*outputs, fromfile="LF input", tofile="CRLF input")))
print("Identical formatting:", outputs[0] == outputs[1])
```

The embedded C++ is a formatting-only fragment; no application dependencies or
compilation are needed. The four style settings reproduce the discrepancy with a
104-column limit.

### Actual result

Both tested versions produce the following diff after output newline normalization:

```diff
--- LF input
+++ CRLF input
@@ -1,11 +1,12 @@
void f()
{
- document += alias ? fmt::format(
- R"(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ document += alias
+ ? fmt::format(
+ R"(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
)",
- name,
- a)
- : fmt::format(R"({})",
- name,
- space_input);
+ name,
+ a)
+ : fmt::format(R"({})",
+ name,
+ space_input);
}
```

The script prints `Identical formatting: False`.

### Expected result

The LF and CRLF inputs should produce the same wrapping and indentation after
normalizing output line endings: an empty diff and `Identical formatting: True`.
The LF layout above fits the configured column limit.

### Additional checks

The discrepancy also reproduces with `LineEnding: LF` or `LineEnding: CRLF` added
to the style, in both tested versions. Each output is stable on a second formatting
pass with the same configuration.

### Possible cause

In 20.1.7,
[`FormatTokenLexer::getNextToken()`](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.7/clang/lib/Format/FormatTokenLexer.cpp#L1373-L1392)
measures a multiline token's first line using the substring before the first LF,
which retains a preceding CR.
The [width helper](https://github.com/llvm/llvm-project/blob/llvmorg-20.1.7/clang/lib/Format/Encoding.h#L42-L77)
falls back to byte length for nonprintable characters. This appears to count the CR
as an extra column when the LF layout's first raw-string line exactly meets the limit.
This explanation is based on source inspection; no patched LLVM build was tested.

### AI assistance

Assisted-by: OpenAI Codex.

AI assisted the investigation, reduction of the example, reproduction script, and
drafting of this report. The reported outputs were captured by executing the stated
formatter binaries.

Contributor guide

Open the contributing guide

Research direction

Reproduce the difference with the provided repro.py and clang-format versions, then inspect FormatTokenLexer::getNextToken() in clang/lib/Format/FormatTokenLexer.cpp and the width helper in clang/lib/Format/Encoding.h. Add regression coverage for equivalent LF and CRLF inputs, and verify that normalized outputs match with the stated style and column limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.