`<print>`: `std::println("{}", ...)` UTF-8 Truncation Bug with `/utf-8` (256-byte Buffer Split)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 11.2k
- Forks
- 1.7k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 22
Description
Describe the bug
When using the MSVC compiler with /utf-8, std::println truncates overly long UTF-8 strings at internal buffer boundaries (replaced with U+FFFD replacement characters) when formatting arguments are used (std::println("{}", str)).
Reproduction Code and Output
#include <print>
int main()
{
std::println("{}", "这是一段超长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长的文本。");
}
Compilation Command: cl.exe /std:c++latest /utf-8 repro.cpp
Compiler Version: 用于 x86 的 Microsoft (R) C/C++ 优化编译器 19.50.35718 版
Expected Behavior: The UTF-8 string is output completely and correctly.
Observed Behavior: 这是一段超长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长���的文本。
Possible Cause
The issue is likely caused by the output mechanism splitting the long string into small, fixed-size chunks (e.g., 256 bytes) before sending them to the console.
The failure is suspected to lie in the specialized handler responsible for committing these chunks to the console: _Fmt_iterator_flush<_Print_to_unicode_console_it>.
This handler, which manages the UTF-8 to Console conversion, appears to simply pass the raw byte chunk's range (_First to _Last) to the underlying write function without ensuring the chunk contains complete UTF-8 characters:
If a chunk ends in the middle of a multi-byte UTF-8 character, committing the incomplete sequence at this point may cause the downstream MultiByteToWideChar conversion to fail, resulting in the observed U+FFFD characters. This suggests the necessary UTF-8 boundary check logic may be missing from this specific specialization before the data is written to the console handle.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Inspect stl/inc/print around lines 107-114, focusing on _Fmt_iterator_flush<Print_to_unicode_console_it>; first reproduce the issue with cl.exe /std:c++latest /utf-8 repro.cpp and trace the chunk handling. Done means the supplied long UTF-8 string prints completely and correctly without U+FFFD replacement characters at the internal buffer boundary.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100