eclipse-jdt / eclipse-jdt/eclipse.jdt.core

InvalidInputException when using Scanner on module import declaration

Open
#5,286 1 comment 0 reactions 1 assignee Claimed by @srikanth-sankaran View on GitHub
Dominant language
Java
Stars
237
Forks
195
Avg merge
1d 12h
Merged PRs (30d)
47

Description

When the Java token scanner gets to an `import module` token, it throws a `InvalidInputException: Unknown token (check Scanner/TerminalToken): TokenNamemodule`.

For example, consider the following test:
```java
@Test
public void testInvalidTokenName() throws InvalidInputException {
String s = """

import module java.base;

int i;
int j;

void main() {
// main
}
""";
IScanner scanner= ToolFactory.createScanner(true, false, false, false);
scanner.setSource(s.toCharArray());
do {
// nothing
} while (scanner.getNextToken() != ITerminalSymbols.TokenNameEOF);
}
```

This test fails with an exception like this:
```
org.eclipse.jdt.core.compiler.InvalidInputException: Unknown token (check Scanner/TerminalToken): TokenNamemodule
at org.eclipse.jdt.internal.compiler.parser.Scanner.invalidToken(Scanner.java:5565)
at org.eclipse.jdt.internal.core.util.PublicScanner.translateTokenToTerminalSymbol(PublicScanner.java:271)
at org.eclipse.jdt.internal.core.util.PublicScanner.getNextToken(PublicScanner.java:118)
at org.eclipse.jdt.text.tests.folding.FoldingTest.testInvalidTokenName(FoldingTest.java:861)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
```

I think the `ITerminalSymbols` class needs a `TokenNameModule` constant and `PublicScanner#translateTokenToTerminalSymbol` should be updated to use that. I guess that method could be refactored to make use of an exhaustive switch so this doesn't happen in the future?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.