RFI: Rebalancing unicode escaping in str::escape_debug for internationalization
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
Triggered by discussion in #158286 @rust-lang/libs-api concluded that our debug escapes are escaping too many characters and are not internationalization-friendly.
E.g. ຄົນສົ່ງໜັງສື currently gets escaped to ຄ\u{ebb}ນສ\u{ebb}\u{ec8}ງໜ\u{eb1}ງສ\u{eb7}
We don't have precise rules ourselves what should and shouldn't be escaped and we'd like to gather input and help from unicode/internationalization experts to find a better balance between various debugging goals and learn what the tradeoffs are.
Once we better understand the tradeoffs and have decided on an approach implementation help, testdata and similar would also be appreciated.
Some desiderata that were brought up during discussion:
- living languages¹ should not be excessively mangled by escaping and remain readable.
A few ligatures getting torn per sentence probably is ok, every other letter getting replaced by an escape wouldn't be - surprising/unexpected/unprintable characters - e.g. a ZWJ or BOM in the middle of ascii strings/latin script - should get escaped
- debug printing shouldn't mess up terminals → control sequences and directionality overrides should remain escaped
- binary size might also be a concern if additional unicode tables are required
¹ "Any text one could pluck from a random international wikipedia page that isn't about ancient typography"
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the discussion in issue #158286 and review the examples and desiderata in this issue. Gather Unicode and internationalization input on which characters str::escape_debug should escape, including readability, terminal safety, and binary-size tradeoffs. Done means there is an agreed set of rules and an implementation approach, with test data identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- internationalization
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100