Tracking Issue for `frontmatter`

This is a tracking issue for the RFC "frontmatter" (rust-lang/rfcs#3503).
The feature gate for the issue is #![feature(frontmatter)].

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
Discussion comments will get marked as off-topic or deleted.
Repeated discussions on the tracking issue may lead to the tracking issue getting locked.

Steps

• Implement the RFC for rustc:
  • #137193
  • rust-lang/rust#140035
  • rust-lang/rust#143708
  • #145751
  • #145754
  • #145766
  • r-a
• Adjust documentation (see instructions on rustc-dev-guide)
  • rust-lang/reference#1974
• Formatting for new syntax has been added to the Style Guide (nightly-style-procedure) (rust-lang/style-team#212)
  • rust-lang/rust#145617
• Stabilization PR (see instructions on rustc-dev-guide)
  • https://github.com/rust-lang/rust/pull/148051

Non-blocking

• rustfmt support

Test coverage prior to stabilization

In the following, the RFC text in its accepted form will be referred to as the "accepted text".

• (Before stabilization) we should add a run-make test that exercises how cargo and rustc works together w.r.t. frontmatter, depending on what the final cooperation scheme we end up using.
• (Before stabilization) we should add a positive smoke test to check that rustfix is able to account for frontmatters (or properly ignore), i.e. //@ run-rustfix.

Unresolved Questions

• What level of quality of error messages is a blocker for stabilization? See also https://github.com/rust-lang/rust/pull/137193#discussion_r1966421271
  • In #140035, the initial (compiler) impl already tries to provide diagnostics for the most common erroneous usages.
  • However, it might be worth refining how the diagnostics responsibility of reporting invalid frontmatters get shared between $tools <-> rustc.
• How does frontmatter interact with -Zunpretty? See discussions in https://github.com/rust-lang/rust/pull/137193#discussion_r1960727886.
  • Implemented in #143708
• How will various tools (rust-analyzer, rustfmt, cargo, clippy) handle frontmatter (and interop w/ rustc)? Will parsing support (or ability to ignore frontmatter) be a stabilization blocker (re. toolchain UX)? For all of them, for a subset of the tools?
• Should frontmatter be outright banned in doctests? Consider these examples: https://github.com/rust-lang/rust/pull/140035#discussion_r2069916501.
• (Needs explicit design decision prior to stabilization) Should whitespace be permitted between the starting dashes and the infostring? E.g. would --- cargo be accepted? In #140035 and the accepted text, whitespace between starting dashes and the infostring is permitted.
• #141367
• Interaction with proc-macro and breakages like #145520

Implementation history

Initial implementations

• #137193
• #140035

(Significant) changes since the RFC was accepted

Frontmatter opener vs infostring starting with - character grammar ambiguity

The accepted text has a grammar ambiguity in the case where the infostring starts with a -.

• The same - character is used to delimit the frontmatter "opener" ---.
• The opener can have more than 3 - characters to support escaping nesting of ---s within the frontmatter.
• However, infostrings described in the accepted text can also begin with -, making it ambiguous as to whether it's a 3-- opener + --starting infostring, or if it is a 4-- opener + infostring.

Remedy: we can require the infostring to begin with Unicode XID_Start, then permit subsequently characters in the set { XID_Continue, . }. I.e. (illustrative)

- infostring = term *
- term       = { all characters - ( whitespace | ',' ) }
+ infostring = XID_Start , { (XID_Continue | '.') } * ;

See discussions at https://github.com/rust-lang/rust/pull/140035#discussion_r2068693754.

• This can be further relaxed in the future (carefully) if it does not introduce more ambiguities.
• RFC initially required infostring to consist of identifier, but was relaxed to support . in e.g. file.ext (see https://github.com/rust-lang/rfcs/pull/3503#discussion_r1481574592).

Illustrative grammar

https://github.com/rust-lang/rust/pull/140035#discussion_r2070377514 (with start and end dashes >= 3 - characters and must match in - count):

frontmatter_start = dashes , { whitespace } * , infostring , { whitespace } * , '\n' ;
frontmatter_end = dashes , { whitespace } * , '\n' ;
infostring = XID_Start , { (XID_Continue | '.') } * ;
dashes = "---" , { '-' } * ;

Start with RFC 3503 and the linked implementation issues and stabilization PR to understand the current frontmatter design. Choose a specific unchecked item rather than working on this tracking issue broadly, then follow its linked issue or pull request. Done means the selected checklist item is resolved with the required implementation, documentation, formatting, or test coverage.