JuliaLang / JuliaLang/JuliaSyntax.jl

Failures parsing General registry

Open
#134 8 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Julia
Stars
293
Forks
50
PR merge metrics
No merged PRs in 30d

Description

Here's a list of failures when parsing General restricted to cases where
* JuliaSyntax.jl fails to parse a file
* the reference parser accepts the file
* or both parsers succeed, but the resulting `Expr`s mismatch

### Parser failures
* [x] `A[x ? y : end]` - AssociativeArrays_0.0.3/src/namedaxis.jl, Reduce_1.2.12/src/Reduce.jl, UnROOT_0.8.1/src/UnROOT.jl
* [x] `function (f()) where T end` - DMRJtensor_0.8.7/lib/Qtensor.jl, DMRJtensor_0.8.7/lib/decompositions.jl, EnglishText_0.6.0/src/numeric.jl, ExprTools_0.1.8/test/function.jl, MemoryMutate_0.0.4/src/MemoryMutate.jl, MultivariatePolynomials_0.4.2/src/differentiation.jl, Singular_0.9.1/src/module/vector.jl, etc why people why??
* [x] `.1..0.1` - DomainSets_0.5.9/test/test_setoperations.jl, `0x01..0xcf` - Rasters_0.2.0/test/sources/gdal.jl
* [x] `import Base.:*` - Empirikos_0.4.5/src/ebayes_targets.jl, `import A.:⋅` - MeasureTheory_0.13.2/test/runtests.jl, Pathnames_0.1.0/src/Pathnames.jl, Porta_0.1.4/src/geometry/biquaternions.jl
* [x] `Base.(:*)` - UnitfulLatexify_1.5.2/src/UnitfulLatexify.jl
* [x] `function ($M).$J() end` - Knet_1.4.9/src/cuarrays/reduction.jl
* [x] `@a@b` - MathematicalSystems_0.11.11/test/@system.jl
* [x] `[a~b]`- ModelingToolkitStandardLibrary_0.1.0/test/test_continuous.jl, NeuralPDE_4.4.0/test/NNPDE_tests.jl, etc
* [x] `@(A)` - NumericalMethodsforEngineers_2.0.0/examples/ch07/WRM/Fig.7.14.jl
* [x] `-1.#comment` - POMDPModels_0.4.15/src/MountainCar.jl
* [x] `A.@a(x).b` - WorkerUtilities_1.0.1/test/runtests.jl
* [x] `(1+2).@x` - julia_1.6.1/test/show.jl

### Parses, but Expr is wrong

* [x] `@.r` - no space after macro name
* [x] `@\ _` - symbol macro names
* [x] `"[1 2\n]"` - hvcat rather than hcat
* [x] `A where {X, Y; Z}` - parameters not in first args slot
* [x] `@a(x; a)` - parameters not in first args slot
* [x] `try a end` should be an error
* [x] `function (f::T{g(i)})() end` and `function (::g(x))() end`
* [x] `.&(x)`
* [x] `+0xff`
* [x] `f(x for a = as, b = bs for c = cs, d = ds)`
* [x] `x ≂ y ≂ z`
* [x] `A[x, y; z = 1]` - parameter block `=` vs `kw`
* [x] `"\"\"\"\n\$x\n b\"\"\""` - wrong deindentation
* [x] `using A: @.` - not lowered to `@__dot__`
* [x] `A{x;y,z}` and `A{x;y,z=1}` - parameter blocks & kw vs =
* [x] `function (:*=(f))() end` - NiLang uses this

### Bizarre forms which parse differently but unclear if it's worth fixing

* [ ] `function (begin end) nothing end` ExprTools_0.1.8/test/function.jl

### Expr conversion failures
* [x] JuliaFormatter_0.22.0/test/files/PProf.jl
* [x] MathOptInterfaceMosek_0.5.2/examples/chainsing.jl
* [x] MosekTools_0.11.0/examples/chainsing.jl
* [x] SDFReader_0.3.4/test/0002.jl

### Bugs in reference parser

* [ ] `export @A @B` should not parse - TSML_2.7.2/src/TSML.jl
* [ ] `17.328679084777833f0` is not correctly rounded?
* [ ] `1.0e+` parses as an `Int128` literal
* [ ] `f()isa(x,y)` is not the juxtaposed multiplication `f()*isa(x,y)`. It's infix `isa` syntax instead.
* [ ] `[i; for i in is]` is allowed by the reference parser but the presence of this semicolon is weird and broken.

The following bugs we can deal with when comparing Exprs

* [x] `0x1.8p23f` is a `Float64` literal, with the trailing `f` ignored (also `0x1p1f0`)
* [x] The macrocall in `"@f(a=1) do\nend"` is not the same as the call in `@f(a=1)`
* [x] `global (x,y)` is the same as `global x,y` in reference parser
* [x] `"\"\"\"\n a\n \n b\"\"\""` parses to `"a\n \nb"` in reference parser - lines with partial whitespace are handled inconsistently
* [x] `(a; b,)` - should be a frankentuple not a block. Occurs a single time in QuadraticFormsMGHyp_0.5.1/src/QuadraticFormsMGHyp.jl and appears to be a syntax mistake.
* [ ] `"\"\"\"\n \$y\n \n \"\"\"\n"` - partial whitespace
* [ ] `"f(g() = x)"` - extra block

### Intentional failures due to being stricter than reference parser

* [x] `import A as (a.b)` - julia_1.6.1/test/syntax.jl
* [ ] Juxtaposition of strings with more keywords is disallowed. For example `"str"for i=is end`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by selecting one unchecked parser or Expr-conversion case from the issue, then inspect the corresponding Julia file path listed there and compare JuliaSyntax.jl with the reference parser. Done means the selected case parses or converts as expected without regressing the other listed cases; the issue does not name a specific parser entry point or test command.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
compilers
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.