dtolnay / dtolnay/proc-macro2

Edition 2021 reserved prefixed tokens are accepted

Open
#534 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
934
Forks
135
PR merge metrics
No merged PRs in 30d

Description

The 2021 edition reserved several forms of prefixed literals that proc-macro2 is currently accepting. I'm not sure what the edition policy of proc-macro2 is, but I would somewhat expect these to be rejected.

```rust
use std::str::FromStr;

fn main() {
for input in [
"prefix#identifier",
"prefix\"string\"",
"prefix'c'",
"prefix#123",
"'prefix#",
] {
println!("{:?}", proc_macro2::TokenStream::from_str(input));
}
}
```

displays the following:

```
Ok(TokenStream [Ident { sym: prefix }, Punct { char: '#', spacing: Alone }, Ident { sym: identifier }])
Ok(TokenStream [Ident { sym: prefix }, Literal { lit: "string" }])
Ok(TokenStream [Ident { sym: prefix }, Literal { lit: 'c' }])
Ok(TokenStream [Ident { sym: prefix }, Punct { char: '#', spacing: Alone }, Literal { lit: 123 }])
Err(LexError { span: Span })
```

The edition chapter on the reservation: https://doc.rust-lang.org/nightly/edition-guide/rust-2021/reserved-syntax.html. This was from [RFC 3101](https://rust-lang.github.io/rfcs/3101-reserved_prefixes.html). The exact grammar from the reference is:

```
RESERVED_TOKEN_DOUBLE_QUOTE ->
IDENTIFIER_OR_KEYWORD _except `b` or `c` or `r` or `br` or `cr`_ `"`

RESERVED_TOKEN_SINGLE_QUOTE ->
IDENTIFIER_OR_KEYWORD _except `b`_ `'`

RESERVED_TOKEN_POUND ->
IDENTIFIER_OR_KEYWORD _except `r` or `br` or `cr`_ `#`

RESERVED_TOKEN_LIFETIME ->
`'` IDENTIFIER_OR_KEYWORD _except `r`_ `#`
```

(Note, proc-macro2 is handling the lifetime tokens correctly.)

Using `proc-macro2 1.0.106`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.