microsoft / microsoft/STL

`<print>`: `std::println("{}", ...)` UTF-8 Truncation Bug with `/utf-8` (256-byte Buffer Split)

Open
#5,894 3 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug format
Dominant language
C++
Stars
11.2k
Forks
1.7k
Avg merge
4d 15h
Merged PRs (30d)
22

Description

Describe the bug

When using the MSVC compiler with /utf-8, std::println truncates overly long UTF-8 strings at internal buffer boundaries (replaced with U+FFFD replacement characters) when formatting arguments are used (std::println("{}", str)).

Reproduction Code and Output

#include <print>

int main()
{
    std::println("{}", "这是一段超长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长的文本。");
}

Compilation Command: cl.exe /std:c++latest /utf-8 repro.cpp

Compiler Version: 用于 x86 的 Microsoft (R) C/C++ 优化编译器 19.50.35718 版

Expected Behavior: The UTF-8 string is output completely and correctly.

Observed Behavior: 这是一段超长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长长���的文本。

Possible Cause

The issue is likely caused by the output mechanism splitting the long string into small, fixed-size chunks (e.g., 256 bytes) before sending them to the console.

The failure is suspected to lie in the specialized handler responsible for committing these chunks to the console: _Fmt_iterator_flush<_Print_to_unicode_console_it>.

This handler, which manages the UTF-8 to Console conversion, appears to simply pass the raw byte chunk's range (_First to _Last) to the underlying write function without ensuring the chunk contains complete UTF-8 characters:

https://github.com/microsoft/STL/blob/5459853bf78ceef1137d845bfeca98f658659691/stl/inc/print#L107-L114

If a chunk ends in the middle of a multi-byte UTF-8 character, committing the incomplete sequence at this point may cause the downstream MultiByteToWideChar conversion to fail, resulting in the observed U+FFFD characters. This suggests the necessary UTF-8 boundary check logic may be missing from this specific specialization before the data is written to the console handle.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect stl/inc/print around lines 107-114, focusing on _Fmt_iterator_flush<Print_to_unicode_console_it>; first reproduce the issue with cl.exe /std:c++latest /utf-8 repro.cpp and trace the chunk handling. Done means the supplied long UTF-8 string prints completely and correctly without U+FFFD replacement characters at the internal buffer boundary.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.