antlr / antlr/antlr4

ErrorListener shouldn't receive a null RecognitionException

Open
#949 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
19k
Forks
3.5k
PR merge metrics
No merged PRs in 30d

Description

I'm trying to write a basic auto-completion tool using ANTLR. In order to determine what types of things to suggest, I'm parsing everything before the cursor and catching the syntax error passed to an error listener to get the expected tokens from the RecognitionException that is passed along to the syntax error.

However, according to the docs, "the RecognitionException is non-null for all syntax errors except when we discover mismatched token errors that we can recover from in-line, without returning from the surrounding rule (via the single token insertion and deletion mechanism)."

I'd argue that this is undesirable behavior because there is no way to get the expected tokens in this case, even though the generated error message passed in the `msg` parameter lists them. I think in these cases it would still be acceptable to throw a InputMismatchException. Consider the following case of a grammar that just excepts two text tokens separated by whitespace:

`start : ALNUM ALNUM EOF`

where `ALNUM` is a string of alphanumeric characters.

When parsing the string `TEST`, the default error listener emits the message `line 1:4 missing ALNUM at ''`, and the RecognitionException is null.

When parsing the string `TEST TEST TEST`, the error message is `line 1:10 extraneous input 'TEST' expecting `, and the RecognitionException is null.

But when parsing the string `TEST *`, the error message is `line 1:5 mismatched input '*' expecting ALNUM`, and the RecognitionException is a InputMismatchException.

It seems like each of these should return an InputMismatchException.

Contributor guide

Open the contributing guide

Research direction

Reproduce the reported grammar with `TEST`, `TEST TEST TEST`, and `TEST *`, then trace the default error listener and inline recovery paths for missing, extraneous, and mismatched tokens. Done means the first two cases provide a non-null InputMismatchException while preserving the reported error messages and behavior for the third case.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.