lark-parser / lark-parser/lark

Optimization: Merge anonymous terminals

Open
#221 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
6k
Forks
523
Avg merge
3d 22h
Merged PRs (30d)
2

Description

It's possible to improve Lark's speed, for all algorithms, by performing grammar simplification. Specifically, it's possible to merge adjacent anonymous terminals together, to reduce the parser work load, without affecting the resulting tree (An example of this technique can be seen in https://github.com/lark-parser/lark/issues/218)

In the same vein, ignored terminals can be joined to anonymous ones, with a similar effect.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the technique described in issue #218 and locating Lark's grammar-construction path. Determine how adjacent anonymous terminals and ignored terminals joined to anonymous terminals can be simplified, then verify that resulting trees are unchanged and parser work is reduced.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.