Unicode support
Open
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 2.9k
- Forks
- 231
- Avg merge
- 19h 55m
- Merged PRs (30d)
- 67
Description
Tokenizer
- add pass to check input files are valid as utf-8
- add util function to check XID_Start/XID_Continue to libcpp
- identifiers
- lifetime label
- raw identifiers
- string literal
- char literal
- Unicode esccape
\uxxxx - loop label
- not implemented yet
- parse whitespaces
- normalize identifiers (including labels) to NFC
- normalize function: https://github.com/Rust-GCC/gccrs/issues/2379
- use it during tokenization: https://github.com/Rust-GCC/gccrs/pull/2489
Parser
- check for
crate_nameattributes- https://doc.rust-lang.org/reference/crates-and-source-files.html#the-crate_name-attribute
- Unicode Alphabetic
- Unicode Numeric (search with
Nl;(decimal digits),Nd;(letter-like numeric), orNo;(other numeric) ) - is_numeric and is_alphabetic functions is added via
- Modify checker
Backend
- punycode (v0 mangling)
Others
- linter
- Fix
#[no_mangle] - Fix legacy name manglling
TODOs
need more tests!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the checklist and its linked issues and pull requests to identify which Unicode work remains; the unchecked items are loop labels and linter support, with a note requesting more tests. Done means completing the remaining scope and adding coverage for the supported Unicode behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, rust
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100