apex-dev-tools / apex-dev-tools/apex-parser

Test CRLF line endings for multiline string literals

Abierto
#158 0 comentarios 0 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Java
Estrellas
38
Forks
12
Merge medio
23 h 56 min
PR fusionados (30 d)
4

Descripción

## Problem

Salesforce known issue [a02g7000008k5RV](https://help.salesforce.com/s/issue?id=a02g7000008k5RV) reports that Apex multiline strings fail to compile when the source is saved with Windows CRLF line endings:

```
Ln 3, Col 35: Unexpected symbol '\r', was expecting '\n'.
Ln 4, Col 13: Unexpected token '{'.
```

The platform's expected behaviour is that multiline strings support LF and CRLF equally, and a fix is understood to be in progress.

Our lexer already accepts CRLF, so we are not exposed — but only incidentally, and nothing tests or documents it.

## Why the current rule works

```antlr
MultilineStringLiteral
: '\'\'\'' [\r\n] ( EscapeSequence | '\'' | ~['\\] )*? '\'\'\''
;
```

`[\r\n]` is a single-character class. On CRLF input the `\r` satisfies the required newline, and the orphaned `\n` is then absorbed by `~['\\]` as ordinary body content. The token lexes and the body is intact.

That is the right outcome reached by accident. The rule comment explains the ordering against `StringLiteral` and the escape semantics, but says nothing about line endings, so a future tightening to `'\r'? '\n'` or `[\n]` — which would look like a correctness improvement — would silently reintroduce the exact platform bug above for every Windows-authored source file, with no test failing.

## Scope

- Lexer/parser tests for a multiline string literal opened with CRLF, asserting the token is recognised and the body content is correct.
- Equivalent test for LF, so the pair documents the intent side by side.
- A test for a bare `\r` opening (classic-Mac endings), recording current behaviour: the character class accepts it. Leniency is the safe direction here, since over-acceptance is harmless while under-acceptance produces a false syntax error on valid code.
- Assert line and column tracking across a CRLF multiline token — issue #102 listed position tracking across multi-line tokens as a task, and CRLF is the case most likely to be off by one.
- Add a line to the rule comment stating that CRLF acceptance is deliberate and covered by tests, so the `[\r\n]` class is not "simplified" later.

## Non-scope

- Changing the grammar rule. Current behaviour is correct; this issue locks it in.
- Runtime newline-stripping semantics for the leading newline.

## No apex-ls change required

Checked alongside this: apex-ls needs nothing.

- `Literal.construct` maps both `Lit.Str` and `Lit.MultiStr` to `StringLiteral` with raw `getText` (`jvm/src/main/scala/com/nawforce/apexlink/cst/Literals.scala:155-165`), so no runtime value is modelled and there is no leading-newline strip to get CRLF-wrong.
- The `Malformed multi-line string literal, the body of '''...''' must start on a new line` diagnostic (`CollectingErrorListener.scala:123`) detects three textually adjacent `StringLiteral` tokens. Because CRLF input lexes as a `MultilineStringLiteral`, that path is unreachable and cannot produce a false positive on Windows files.

One residual edge, noted but not proposed for action: since `StringLiteral` retains raw text, two textually identical multiline strings saved with different line endings are unequal values. Only reachable where literal text is compared, e.g. duplicate `when` detection in switch, which does accept `MultilineStringLiteral` (`cst/stmts/Switch.scala:83`).

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start with the existing lexer/parser tests and the MultilineStringLiteral rule. Add side-by-side LF and CRLF cases, record bare-CR behavior, and verify line and column tracking across the token; then update the rule comment to preserve the deliberate acceptance. Done means the tests confirm token recognition, body content, and positions without changing the grammar or runtime newline handling.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
java
Área
compilers, testing-qa
Tipo de issue
Refactorización
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Activo
Claridad
Bien especificado
Aptitud para principiantes
78/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.