JuliaLang / JuliaLang/JuliaSyntax.jl

Incorrect parsing of `import ..+`

Open
#235 3 comments 0 reactions 0 assignees View on GitHub
bug intro issue parser tokenizer
Dominant language
Julia
Stars
293
Forks
50
PR merge metrics
No merged PRs in 30d

Description

Import paths with a prefixed pair of dots should parse as follows:

```julia
julia> JuliaSyntax.parse(SyntaxNode, "import ..A")
line:col│ tree │ file_name
1:1 │[import]
1:7 │ [.]
1:8 │ .
1:9 │ .
1:10 │ A
```

But for operators this doesn't work

```julia
julia> JuliaSyntax.parse(SyntaxNode, "import ..+")
line:col│ tree │ file_name
1:1 │[import]
1:7 │ [.]
1:8 │ ..+
```

This appears to be because the initial tokenize step picks the import path up as part of an invalid operator

```julia
julia> collect(JuliaSyntax.Tokenize.tokenize("..-"))
2-element Vector{JuliaSyntax.Tokenize.RawToken}:
0-2 ErrorInvalidOperator
3-2 EndMarker
```

Which is normally correct, but not helpful within import paths.

The reference parser also has this bug:

```
julia> dump(Meta.parse("import ..+"))
ERROR: Base.Meta.ParseError("invalid operator \"..+\" near column 9")
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the issue with JuliaSyntax.parse(SyntaxNode, "import ..+") and inspect JuliaSyntax.Tokenize.tokenize("..-") first. Trace how the tokenizer and parser handle prefixed dots in import paths, using the shown ..A and ..+ trees as the expected comparison. Done means the operator form parses consistently with the prefixed-dot import path and the relevant behavior is covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.