Fallback `Ident` is not normalized
- Dominant language
- Rust
- Stars
- 934
- Forks
- 135
- PR merge metrics
- No merged PRs in 30d
Description
Starting from Rust 1.42.0 (rust-lang/rust#66670) Unicode identifiers are normalized in NFC form, that means the following proc-macro will produce `true`:
```rust
extern crate proc_macro;
use proc_macro::{Ident, Span, TokenStream};
#[proc_macro]
pub fn check(_: TokenStream) -> TokenStream {
let decomposed = Ident::new("e\u{301}", Span::call_site());
let composed = Ident::new("\u{e9}", Span::call_site());
(composed.to_string() == decomposed.to_string()).to_string().parse().unwrap()
}
```
However, the `proc_macro2` fallback does not perform NFC normalization, meaning the equivalent code in `proc_macro2` will produce `false`:
```rust
use proc_macro2::{Ident, Span, TokenStream};
pub fn check(_: TokenStream) -> TokenStream {
let decomposed = Ident::new("e\u{301}", Span::call_site());
let composed = Ident::new("\u{e9}", Span::call_site());
(composed.to_string() == decomposed.to_string()).to_string().parse().unwrap()
}
fn main() {
dbg!(check(TokenStream::default()));
// Ident { sym: false, span: bytes(1..6) }
}
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.