microsoft / microsoft/STL

`<print>`: Investigate optimizing for UTF-8 terminals

Open
#5,134 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

performance
Dominant language
C++
Stars
11.2k
Forks
1.7k
Avg merge
4d 15h
Merged PRs (30d)
22

Description

At the November 2024 meeting, LWG-4044 "Confusing requirements for std::print on POSIX platforms" was resolved in the C++ Working Paper.

In https://github.com/microsoft/STL/issues/5109#issuecomment-2496252382, @CaseyCarter explained:

There exist terminals that can only display Unicode via a "native Unicode API", and terminals that can display Unicode simply via the normal means of dumping output. The intent of the issue is that only the former require the determination if a stream refers to such a terminal, and require flushing the "normal" output before invoking such a "native Unicode API" to output Unicode.

In real world terms, "native Unicode API" means "WriteConsoleW on Windows". Non-Windows platforms are supposed to ignore these instructions but it wasn't clear to them that they should, so the issue is trying to clarify that intent.

Of course there are also terminals on Windows that are capable of displaying Unicode via the normal output mechanisms. My machine with the "Beta: Use UTF-8 for language support" feature activated defaults to codepage 65001 in terminals and doesn't need "native Unicode API" to output Unicode. At some point we should start detecting this condition and avoiding the mess of flushing stdout and transcoding to UTF-16 to call WriteConsoleW.

TLDR: We don't need to do anything for correctness with regard to LWG-4044, we may want to make changes for performance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the std::print Windows output path and investigate how the terminal's UTF-8 capability or code page relates to use of WriteConsoleW. Done means identifying when normal output can display Unicode and avoiding unnecessary stdout flushing and UTF-8-to-UTF-16 conversion in that case.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.