apex-dev-tools / apex-dev-tools/apex-parser

Test CRLF line endings for multiline string literals

Aperta
#158 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Java
Stelle
38
Fork
12
Merge medio
23h 56m
PR unite (30g)
4

Descrizione

## Problem

Salesforce known issue [a02g7000008k5RV](https://help.salesforce.com/s/issue?id=a02g7000008k5RV) reports that Apex multiline strings fail to compile when the source is saved with Windows CRLF line endings:

```
Ln 3, Col 35: Unexpected symbol '\r', was expecting '\n'.
Ln 4, Col 13: Unexpected token '{'.
```

The platform's expected behaviour is that multiline strings support LF and CRLF equally, and a fix is understood to be in progress.

Our lexer already accepts CRLF, so we are not exposed — but only incidentally, and nothing tests or documents it.

## Why the current rule works

```antlr
MultilineStringLiteral
: '\'\'\'' [\r\n] ( EscapeSequence | '\'' | ~['\\] )*? '\'\'\''
;
```

`[\r\n]` is a single-character class. On CRLF input the `\r` satisfies the required newline, and the orphaned `\n` is then absorbed by `~['\\]` as ordinary body content. The token lexes and the body is intact.

That is the right outcome reached by accident. The rule comment explains the ordering against `StringLiteral` and the escape semantics, but says nothing about line endings, so a future tightening to `'\r'? '\n'` or `[\n]` — which would look like a correctness improvement — would silently reintroduce the exact platform bug above for every Windows-authored source file, with no test failing.

## Scope

- Lexer/parser tests for a multiline string literal opened with CRLF, asserting the token is recognised and the body content is correct.
- Equivalent test for LF, so the pair documents the intent side by side.
- A test for a bare `\r` opening (classic-Mac endings), recording current behaviour: the character class accepts it. Leniency is the safe direction here, since over-acceptance is harmless while under-acceptance produces a false syntax error on valid code.
- Assert line and column tracking across a CRLF multiline token — issue #102 listed position tracking across multi-line tokens as a task, and CRLF is the case most likely to be off by one.
- Add a line to the rule comment stating that CRLF acceptance is deliberate and covered by tests, so the `[\r\n]` class is not "simplified" later.

## Non-scope

- Changing the grammar rule. Current behaviour is correct; this issue locks it in.
- Runtime newline-stripping semantics for the leading newline.

## No apex-ls change required

Checked alongside this: apex-ls needs nothing.

- `Literal.construct` maps both `Lit.Str` and `Lit.MultiStr` to `StringLiteral` with raw `getText` (`jvm/src/main/scala/com/nawforce/apexlink/cst/Literals.scala:155-165`), so no runtime value is modelled and there is no leading-newline strip to get CRLF-wrong.
- The `Malformed multi-line string literal, the body of '''...''' must start on a new line` diagnostic (`CollectingErrorListener.scala:123`) detects three textually adjacent `StringLiteral` tokens. Because CRLF input lexes as a `MultilineStringLiteral`, that path is unreachable and cannot produce a false positive on Windows files.

One residual edge, noted but not proposed for action: since `StringLiteral` retains raw text, two textually identical multiline strings saved with different line endings are unequal values. Only reachable where literal text is compared, e.g. duplicate `when` detection in switch, which does accept `MultilineStringLiteral` (`cst/stmts/Switch.scala:83`).

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start with the existing lexer/parser tests and the MultilineStringLiteral rule. Add side-by-side LF and CRLF cases, record bare-CR behavior, and verify line and column tracking across the token; then update the rule comment to preserve the deliberate acceptance. Done means the tests confirm token recognition, body content, and positions without changing the grammar or runtime newline handling.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
compilers, testing-qa
Tipo di issue
Refactoring
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Attiva
Chiarezza
Specificata chiaramente
Idoneità per principianti
78/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.