tokeniser: add `TokenKind::Newline` to disambiguate statement positions
- Dominant language
- Rust
- Stars
- 28
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
For cases such as `t: f() = x;` the parser produces two statements, the first being the `t: f`, and the second being `() = x` which is
technically valid to do, however it is very unclear from the user's perspective what this means. Ideally, the parser should fail in this instance since the statement is all on one line making it unclear what this means.
To solve this problem, a new token kind `Newline` should be introduced to aid the parser in dealing with this scenario. The tokeniser would treat a consequitve trail of newlines as a single one (even with spaces in between).
Then, when a token stream is located on a single line, it is treated as a single line statement. The exclusion to the rule are any kind of tokens that can combine multiple expressions together, i.e. binary operators, or namespace accesses `::` and `.`, and possibly others.
Contributor guide
Assessment
This issue has not been assessed yet.