Decide what to do about the places where we strip shebang but not frontmatter
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.2k
- PR merge metrics
- PR metrics pending
Description
This issue keeps track of the following FIXME so it doesn't get forgotten (esp. since this could be relevant for the next edition):
Us not stripping frontmatter (but shebang) in source_str_to_stream for backward compatibility reasons (https://github.com/rust-lang/rust/issues/145520) means (among other things) that we don't strip frontmatter in proc_macro_server::Rustc::ts_from_str aka proc_macro::TokenStream::from_str across the bridge.
As a reminder, RFC 3503 | frontmatter set a goal to strip frontmatter everywhere where shebang is stripped:
This applies anywhere shebang stripping is performed. For example, if include! strips shebangs, then it will also frontmatter.
Two potential solutions:
- Start stripping frontmatter in
TokenStream::from_strin the next edition - Stop stripping shebang in
TokenStream::from_str(†)
(†) Reasoning: What TokenStream::from_str receives represents arbitrary Rust source code (if lexically valid) like (parts of) expressions, patterns, types, items, bounds, etc. Therefore, I would say it's not really understood to be taking an entire source file where it would make sense to perform all transformations (BOM removal, CRLF "normalization", shebang+frontmatter stripping). As a another datapoint, we've recently stopped stripping shebang(+frontmatter) in expression-context include!(…)s (contrary to item context), https://github.com/rust-lang/rust/pull/146377, for a similar reason.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the FIXME in compiler/rustc_parse/src/lib.rs and inspect source_str_to_stream alongside proc_macro_server::Rustc::ts_from_str and TokenStream::from_str. Read RFC 3503, issue 145520, and PR 146377 to compare the two proposed behaviors. Done means a decided and consistently implemented treatment of shebang and frontmatter in the affected paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100