antlr / antlr/antlr4

New feature: parser rule inlining

Open
#1,963 6 comments 11 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
19k
Forks
3.5k
PR merge metrics
No merged PRs in 30d

Description

Originally submitted as a question at [StackOverflow](https://stackoverflow.com/questions/45191221/is-there-a-parser-equivalent-of-fragment-marking-in-antlr4)

Make 'fragment' marking work not just for lexer rules but for parser as well.
F.e.:
```
file: ( item | class_decl )*;
class_decl: 'class' class_name '{' type_decl* data_decl* code_decl* '}';
type_decl: 'typedef' ('bool'|'int'|'real') type_name;
const_decl: 'const' type_name const_name;
var_decl: 'var' type_name var_name;
...
fragment item: type_decl | data_decl | code_decl;
fragment data_decl: const_decl | var_decl;
fragment code_decl: function_decl | procedure_decl;
fragment class_name: ID;
fragment type_name: ID;
fragment const_name: ID;
fragment var_name: ID;
```
ANTLR should turn above into the following before generation of parser:
```
file: ( type_decl | const_decl | var_decl | function_decl | procedure_decl | class_decl )*;
class_decl: 'class' ID '{' type_decl* ( const_decl | var_decl )* ( function_decl | procedure_decl )* '}';
type_decl: 'typedef' ('bool'|'int'|'real') ID;
const_decl: 'const' ID ID;
var_decl: 'var' ID ID;
...
```
Reasoning behind this feature:
The rules marked as fragment are there for clarity/documentation and reusability, however from syntax point of view it is f.e. really a `var_decl` that is actual direct element of `file` or `class_decl` and there should be a way to reflect that in content of contexts created by the parser. All the intermediate contexts that are now created for `item`, `data_decl` etc. are superfluous, needlessly take space and make it so visitor is bound to organizational structure of the grammar instead of its actual meaning.

Contributor guide

Open the contributing guide

Research direction

Begin with the linked StackOverflow question and the grammar examples in the issue. Determine how parser rules marked as fragments should be expanded before generation, and verify that intermediate parser contexts are removed while the shown grammar structure and visitor-facing contexts match the requested result.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.