JuliaLang / JuliaLang/JuliaSyntax.jl

Newlines handled differently compared to old parser

Open
#316 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
293
Forks
50
PR merge metrics
No merged PRs in 30d

Description

I am not really sure if this is a bug, or maybe it's a bug in the original parser, but it looks like newlines are handled differently in JuliaSyntax.

This broke Documenter's tests (for some reason it's blacklisted in PkgEval), since we do some exact string checks there for parsed sub-expressions, so the cursor value changes threw those tests off. It's not a problem to handle this with version guards (https://github.com/JuliaDocs/Documenter.jl/pull/2146), but it would be good to know whether this change might change back at some point or not.

Here are some `Meta.parse` calls in 1.9 vs today's master:

```
julia@1.9.1> Meta.parse("x += 3\nγγγ_γγγ\nγγγ\n\n", 22; raise=true)
(:γγγ, 29)

julia@1.10.0-DEV.1556> Meta.parse("x += 3\nγγγ_γγγ\nγγγ\n\n", 22; raise=true)
(:γγγ, 30)
```

```
julia@1.9.1> Meta.parse("x = Int[]\r\n\r\npush!(x, 1)\r\n\n", 14)
(:(push!(x, 1)), 27)

julia@1.10.0-DEV.1556> Meta.parse("x = Int[]\r\n\r\npush!(x, 1)\r\n\n", 14)
(:(push!(x, 1)), 28)
```

```
julia@1.9.1> Meta.parse("x = Int[]\n\npush!(x, 1)\n\n", 12)
(:(push!(x, 1)), 24)

julia@1.10.0-DEV.1556> Meta.parse("x = Int[]\n\npush!(x, 1)\n\n", 12)
(:(push!(x, 1)), 25)
```

```
julia@1.9.1> Meta.parse("1 + 1\n2 + 2\n\n", 7)
(:(2 + 2), 13)

julia@1.10.0-DEV.1556> Meta.parse("1 + 1\n2 + 2\n\n", 7)
(:(2 + 2), 14)
```

```
julia@1.9.1> Meta.parse("1 + 1\n\n\n", 1)
(:(1 + 1), 7)

julia@1.10.0-DEV.1556> Meta.parse("1 + 1\n\n\n", 1)
(:(1 + 1), 9)
```

Slightly more interesting case with a comment line:

```
julia@1.9.1> Meta.parse("1 + 1\n# comment\n\n", 1)
(:(1 + 1), 7)

julia@1.10.0-DEV.1556> Meta.parse("1 + 1\n# comment\n\n", 1)
(:(1 + 1), 18)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.