rust-lang / rust-lang/rust

Should tokens passed through `macro_rules` be able to join?

Open
#127,123 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-macros A-proc-macro-back-compat C-discussion T-compiler T-lang WG-macros
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

Should this example

macro_rules! mac {
    ($a:tt sep $b:tt) => { 1 $a$b 2 } 
}

fn main() {
    mac!(= sep =);
}

emit 1 == 2 and compile successfully?
Or, in other words, should the first emitted = have the Spacing::Joint set, if the output is processed by a proc macro?

We need some well-defined rule for setting spacing for tokens produced from macro variables.

Possible alternatives:

    • Always use Joint.
      • This is very unreasonable.
    • Always use Alone.
      • This is the first reasonable alternative.
    • Inherit from the passed token - that means Alone in the example above because there is a space after the first = in mac!(= sep =).
      • Probably unreasonable, the token sequence in the input have little relation to the output sequence for which we emit the spacing, in the example above it is demonstrated by sep being present in the input but not in the output.
      • Note: the issue is about single passed token trees (or last token trees in sequences), tokens passed in the middle of something like $($tt)* should of course use this option.
    • Inherit from the macro parameter declaration - i.e. $a:tt in the example (or just the $a part of it).
      • This doesn't seem reasonable to me
    • Inherit from the macro parameter use, that means Joint in the example above because there is no space after $a in 1 $a$b 2.
      • This is the second reasonable alternative.

Right now the spacing is emitted inconsistently, sometimes it follows the rule 2., sometimes 3..
Adopting any consistent rule may cause breakage (@nnethercote may remember which exactly) and needs to go through crater.

  • I would personally try the rule 5. first, because it gives macro author freedom to control and change spacing as they want, unlike rule 2. that prevents joining entirely.
  • However we may be forced to adopt rule 2. instead due to backward compatibility issues. It would be less flexible, but macro variables creating a "barrier" for joining is also a reasonable enough model.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the macro_rules example and the proc_macro Spacing::Joint documentation linked in the issue. Compare the proposed spacing rules, identify the existing inconsistent behavior, and use crater to assess compatibility before selecting and specifying one consistent rule.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.