rust-lang / rust-lang/fls

Problems with the syntax of lifetimes and labels

Open
#582 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-lexer
Dominant language
reStructuredText
Stars
497
Forks
41
Avg merge
3h 52m
Merged PRs (30d)
4

Description

The FLS's grammar defines a lifetime as follows:

§4.14.1 (fls_yqcygq3y6m5j):

Lifetime ::=
    ' (NonKeywordIdentifier | RawIdentifierKeyword)

and a label as follows:

§6.15.5 (fls_uusi0zej55is)

   LabelIndication ::=
       ' NonKeywordIdentifier
Problem 1

In Rust, whitespace is not permitted here between the ' and the identifier, but nothing in the grammar indicates that the usual rule about whitespace is suspended here.

Problem 2

The LexicalElement production in §2.2 (fls_fgnllgz5k3e6) doesn't allow '.

The only LexicalElement that can begin with ' is CharacterLiteral.

That means that

  • TokenTree* doesn't accept 'a
  • so MacroInvocation doesn't accept m!('a)

But Rust accepts that macro invocation.

(Perhaps the intent was to include ' in the Punctuation production, but it isn't there.)

Problem 3

Possibly more seriously, I don't think a grammar that tries to treat 'a as made up of two separate lexical elements can be made to work.

Consider the following macro invocation: m!('r"a").

In Rust the macro will see two tokens: 'r and "a".

A model which allows ' to stand alone at lexing time is going to end up analysing this as
' followed by a raw string literal r"a".

Note this is a place where the FLS has diverged from the Reference, which defines a LIFETIME_TOKEN and includes it in its equivalent of LexicalElement.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the FLS grammar sections §2.2 (LexicalElement), §4.14.1 (Lifetime), and §6.15.5 (LabelIndication), then compare their treatment with the Rust Reference's LIFETIME_TOKEN. Verify the grammar against m!('a) and m!('r"a"); done means lifetime and label syntax, whitespace, and macro tokenization are specified consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.