JuliaLang / JuliaLang/JuliaSyntax.jl

Parsing large malformed files could be faster

Open
#345 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
293
Forks
50
PR merge metrics
No merged PRs in 30d

Description

Parsing large, completely malformed files takes a long time. I think it'd probably make sense to detect "parsing this file is a complete disaster, it's probably not even valid Julia syntax" somehow and bail out if we can. We still need to respect the `parseall()` API guarantee that the entire file will be covered with a syntax tree, however.

Upstream issue showing how this can be a bad user experience: https://github.com/JuliaLang/julia/issues/50799

Example:

```julia
julia> download("https://www.gutenberg.org/cache/epub/1727/pg1727.txt", "pg1727.txt");

julia> using JuliaSyntax

julia> str = read("pg1727.txt", String);

julia> @time try parseall(Expr, str^1000) catch ; end
46.374221 seconds (25.20 M allocations: 4.515 GiB, 0.50% gc time, 0.06% compilation time)

julia> length(str^1000)/1e6
710.502
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.