apex-dev-tools / apex-dev-tools/apex-parser

Test CRLF line endings for multiline string literals

Offen
#158 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Java
Sterne
38
Forks
12
Ø Merge
23 Std. 56 Min.
Gemergte PRs (30 T.)
4

Beschreibung

## Problem

Salesforce known issue [a02g7000008k5RV](https://help.salesforce.com/s/issue?id=a02g7000008k5RV) reports that Apex multiline strings fail to compile when the source is saved with Windows CRLF line endings:

```
Ln 3, Col 35: Unexpected symbol '\r', was expecting '\n'.
Ln 4, Col 13: Unexpected token '{'.
```

The platform's expected behaviour is that multiline strings support LF and CRLF equally, and a fix is understood to be in progress.

Our lexer already accepts CRLF, so we are not exposed — but only incidentally, and nothing tests or documents it.

## Why the current rule works

```antlr
MultilineStringLiteral
: '\'\'\'' [\r\n] ( EscapeSequence | '\'' | ~['\\] )*? '\'\'\''
;
```

`[\r\n]` is a single-character class. On CRLF input the `\r` satisfies the required newline, and the orphaned `\n` is then absorbed by `~['\\]` as ordinary body content. The token lexes and the body is intact.

That is the right outcome reached by accident. The rule comment explains the ordering against `StringLiteral` and the escape semantics, but says nothing about line endings, so a future tightening to `'\r'? '\n'` or `[\n]` — which would look like a correctness improvement — would silently reintroduce the exact platform bug above for every Windows-authored source file, with no test failing.

## Scope

- Lexer/parser tests for a multiline string literal opened with CRLF, asserting the token is recognised and the body content is correct.
- Equivalent test for LF, so the pair documents the intent side by side.
- A test for a bare `\r` opening (classic-Mac endings), recording current behaviour: the character class accepts it. Leniency is the safe direction here, since over-acceptance is harmless while under-acceptance produces a false syntax error on valid code.
- Assert line and column tracking across a CRLF multiline token — issue #102 listed position tracking across multi-line tokens as a task, and CRLF is the case most likely to be off by one.
- Add a line to the rule comment stating that CRLF acceptance is deliberate and covered by tests, so the `[\r\n]` class is not "simplified" later.

## Non-scope

- Changing the grammar rule. Current behaviour is correct; this issue locks it in.
- Runtime newline-stripping semantics for the leading newline.

## No apex-ls change required

Checked alongside this: apex-ls needs nothing.

- `Literal.construct` maps both `Lit.Str` and `Lit.MultiStr` to `StringLiteral` with raw `getText` (`jvm/src/main/scala/com/nawforce/apexlink/cst/Literals.scala:155-165`), so no runtime value is modelled and there is no leading-newline strip to get CRLF-wrong.
- The `Malformed multi-line string literal, the body of '''...''' must start on a new line` diagnostic (`CollectingErrorListener.scala:123`) detects three textually adjacent `StringLiteral` tokens. Because CRLF input lexes as a `MultilineStringLiteral`, that path is unreachable and cannot produce a false positive on Windows files.

One residual edge, noted but not proposed for action: since `StringLiteral` retains raw text, two textually identical multiline strings saved with different line endings are unequal values. Only reachable where literal text is compared, e.g. duplicate `when` detection in switch, which does accept `MultilineStringLiteral` (`cst/stmts/Switch.scala:83`).

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Rechercherichtung

Start with the existing lexer/parser tests and the MultilineStringLiteral rule. Add side-by-side LF and CRLF cases, record bare-CR behavior, and verify line and column tracking across the token; then update the rule comment to preserve the deliberate acceptance. Done means the tests confirm token recognition, body content, and positions without changing the grammar or runtime newline handling.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
java
Bereich
compilers, testing-qa
Issue-Typ
Refactoring
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Aktiv
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
78/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.