rust-lang / rust-lang/rust

RFI: Rebalancing unicode escaping in str::escape_debug for internationalization

Open
#159,694 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-str A-Unicode C-enhancement E-help-wanted T-libs
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Triggered by discussion in #158286 @rust-lang/libs-api concluded that our debug escapes are escaping too many characters and are not internationalization-friendly.
E.g. ຄົນສົ່ງໜັງສື currently gets escaped to ຄ\u{ebb}ນສ\u{ebb}\u{ec8}ງໜ\u{eb1}ງສ\u{eb7}

We don't have precise rules ourselves what should and shouldn't be escaped and we'd like to gather input and help from unicode/internationalization experts to find a better balance between various debugging goals and learn what the tradeoffs are.
Once we better understand the tradeoffs and have decided on an approach implementation help, testdata and similar would also be appreciated.

Some desiderata that were brought up during discussion:

  • living languages¹ should not be excessively mangled by escaping and remain readable.
    A few ligatures getting torn per sentence probably is ok, every other letter getting replaced by an escape wouldn't be
  • surprising/unexpected/unprintable characters - e.g. a ZWJ or BOM in the middle of ascii strings/latin script - should get escaped
  • debug printing shouldn't mess up terminals → control sequences and directionality overrides should remain escaped
  • binary size might also be a concern if additional unicode tables are required

¹ "Any text one could pluck from a random international wikipedia page that isn't about ancient typography"

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the discussion in issue #158286 and review the examples and desiderata in this issue. Gather Unicode and internationalization input on which characters str::escape_debug should escape, including readability, terminal safety, and binary-size tradeoffs. Done means there is an agreed set of rules and an implementation approach, with test data identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
internationalization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.