Macros are not correctly parsed

I tried this code:

macro_rules! itemizer {
    () => {
        struct S;

        impl S {
            fn new() -> S { S }
        }
    }
}

I expected to see this happen:

Compiles correctly, but the AST should consider it as "not semicoloned".

Instead, this happened:

It compiles correctly but is considered to have a semicolon. Therefore the tokenstream dumper cannot rely on "is_semicoloned" to guess wether a semicolon shall output.

A simple fix would be rust-parse-impl.h line 1801

      return AST::MacroInvocation::Regular (std::move (invoc_data),
					    std::move (outer_attrs),
-					    macro_locus, true);
+					    macro_locus, false);

But it breaks the testsuite since the semicolon status is relied on with a different behavior during the macro expansion.

Meta

• e7ea5fb64026c72f4f93e1ecd92ce7e5d2941b54

Start at rust-parse-impl.h line 1801 and inspect AST::MacroInvocation::Regular, then run the affected testsuite cases around macro expansion and semicolon handling. Trace how the tokenstream dumper uses is_semicoloned. Done means the AST records the macro invocation's semicolon status correctly without breaking the existing testsuite behavior during expansion.