wavefnd / wavefnd/Wave

Make CR-only source line endings consistent across lexer tracking and literals

Open
#598 0 comments 0 reactions 0 assignees View on GitHub
bug cross-platform help wanted needs testing
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

The lexer recognizes carriage return (`\\r`) as whitespace, but only line feed (`\\n`) advances the logical source line.

In `skip_whitespace`, `\\r` is consumed together with ordinary horizontal whitespace, while `\\n` increments `line` and updates `line_start`.

The literal scanner also treats only `\\n` as a raw source newline terminating a string. A lone `\\r` can therefore be consumed as content even though the same character is treated as source whitespace outside the literal.

This creates inconsistent behavior for CR-only source files:

- token line numbers do not advance at `\\r`,
- columns continue from the previous physical line,
- raw CR inside a string does not take the same newline path as LF,
- diagnostics can disagree with the physical source layout.

CRLF must remain a single newline rather than becoming two logical lines.

Code evidence:

- `front/lexer/src/trivia.rs` — `skip_whitespace`
- `front/lexer/src/literals.rs` — string literal newline handling
- lexer diagnostic line/column tracking

Acceptance:

- [ ] Define source newline handling for LF, CRLF, and lone CR.
- [ ] Count CRLF as exactly one logical newline.
- [ ] Handle lone CR consistently in token tracking, comments, and literals.
- [ ] Add lexer tests comparing equivalent LF, CRLF, and CR-only sources.
- [ ] Verify diagnostic line and column locations under all three encodings.
- [ ] Avoid changing escape-sequence `\\r` semantics inside string/char literals.

Contributor guide

Open the contributing guide

Research direction

Start in front/lexer/src/trivia.rs and front/lexer/src/literals.rs, then trace the lexer diagnostic line/column tracking. Compare LF, CRLF, and lone-CR inputs through whitespace, comments, and literals while preserving escaped \r behavior. Done means each physical newline form produces consistent tokens and diagnostics, with CRLF counted once.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.