hash-org / hash-org/hashc

tokeniser: add `TokenKind::Newline` to disambiguate statement positions

Open
#786 0 comments 0 reactions 1 assignee Claimed by @feds01 View on GitHub
parser
Dominant language
Rust
Stars
28
Forks
1
PR merge metrics
No merged PRs in 30d

Description

For cases such as `t: f() = x;` the parser produces two statements, the first being the `t: f`, and the second being `() = x` which is
technically valid to do, however it is very unclear from the user's perspective what this means. Ideally, the parser should fail in this instance since the statement is all on one line making it unclear what this means.

To solve this problem, a new token kind `Newline` should be introduced to aid the parser in dealing with this scenario. The tokeniser would treat a consequitve trail of newlines as a single one (even with spaces in between).

Then, when a token stream is located on a single line, it is treated as a single line statement. The exclusion to the rule are any kind of tokens that can combine multiple expressions together, i.e. binary operators, or namespace accesses `::` and `.`, and possibly others.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.